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A FORMULA FOR CERTAIN SHALIKA GERMS OF RAMIFIED 

UNITARY GROUPS 

CHENG-CHIANG TSAI 


Abstract. In this article, for nilpotent orbits of ramified quasi-split unitary groups with 
two Jordan blocks, we give closed formulas for their Shalika germs at certain equi-valued 
elements with half-integral depth previously studied by Hales [3- These elements are 
parametrized by hyperelliptic curves defined over the residue field, and the numbers we 
obtain can be expressed in terms of Frobenius eigenvalues on the Aadic of the curves, 
generalizing previous result of Hales on stable subregular Shalika germs. These Shalika 
germ formulas imply new results on stability and endoscopic transfer of nilpotent orbital 
integrals of ramified unitary groups. We mention also how the same numbers appear in 
the local character expansion of specific supercuspidal representations and consequently 
dimensions of degenerate Whittaker models. 
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1. Introduction 

We begin by introducing the unitary groups, related algebraic groups, Lie algebras and 
representations, and then the notion of Shalika germs. After that we can state our Shalika 
germ formulas, and describe its applications. 

Let F be a non-archimedean local field and k its residue field. We fix an algebraic closure 
k of k. We assume char(A;) 7 ^ 2. Let F be a ramified quadratic extension over F. Note 
E/F is tame. Fix in this article a uniformizer vr G F whose square root G F. Let 
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n > 1 be an integer and let G = Un{E/F) be the quasi-split unitary group of n variables 
over F which splits over F. We also assume either char(F) = 0 or char(F) > n. 

The reason for the notation G is that we prefer to, just like Bruhat-Tits and in geometric 
Langlands, think of reductive groups over F intuitively as an ind-pro-scheme over the 
residue field k. For this reason, in this article everything - groups, Lie algebras and their 
elements - that lives over F will have its notation with a tilde ~. 

Fix a vertex x on the Bruhat-Tits building of G over F whose reductive quotient is 
SO„(A;). The vertex x becomes hyperspecial after base change to F. The reductive quotient 
at X over F is (the /c-points of) G := GL„. The root system of G is in canonical bijection 
with the root system of G/e, and we can choose compatible pinnings for G and G/e- The 
non-trivial element in Gal{F/F) then provides an involution 0 on G such that the reductive 
quotient at x over F is (G^)°(fe) = SOn{k). A detailed and general construction of this is 
described in [T5l Sec. 4], 

Write g = Lie G and g = Lie G. The involution 9 also acts on g. We’ll write G(0) = 
[G^)° = SOn/fc, g(0) = g®^^, and g(l) = This provides a Z/2-grading on g. Write V 

for the n-dimensional standard representation of G(0) and g(0). We have g(l) = Sym^(l/) 
as G(0)-representations. The Moy-Prasad filtration at x jumps at half-integral numbers, 
and satisfies G(F)x,o/G(F)x,i /2 = G(0)(k), g(T)x,d/ 2 /g(P')x,(d+i )/2 = Vd G Z, 

where the latter isomorphism is as a G( 0 )(A:)-representation. 

Write g(l)^^ := g^^ H g(l) where g^^ is the open subset of regular semisimple elements 
in the Lie algebra. Fix from now on a T G g(l)^^(A;). We can see T as a self-adjoint 
endomorphism on V. The monic characteristic polynomial pT is a separable polynomial of 
degree n. Consequently Gt '■= {y^ = Pt{x)) is a hyperelliptic curve with genus g = 

In fact, the representation G(0) r\ g(l) was first considered by Bhargava-Gross [2] for the 
study of arithmetic statistics about these hyperelliptic curves. 

Consider the quotient map g(F’)x,_i /2 ^ 0(1)(^)- Let T G g(T)x,_i /2 be any lift. Such 
a T is always regular semisimple and elliptic, i.e. Stabg(r) is an anisotropic torus over F. 
In fact, it’s even anisotropic over This implies that the orbits in the stable orbit of T 
enjoys a bijection with the orbits in the stable orbit of T ; see Lemma 14.51 (The notion of 
stable orbit is reviewed in Section [5J) 

Denote by 0(0) the set of nilpotent orbits in g(T), and J{X,f) the orbital integral of 
/ on the orbit of Ai G g. We will often identify an element in g(F’) with its orbit when 
talking about orbital integrals and Shalika germs. The theorem of Shalika m asserts, for 
char(F’) = 0 or char(F’) ;§> 0, the existence of constants, the Shalika germs Fc)(r) G Q 
such that 

(1.1) J(T,/)= Toif)JiOJ), 

OeO{o) 

for any compactly supported functions / on g(F’) that are locally constant by a sufficiently 
large lattice. 

In this article, we prove the following theorem for Shalika germs of nilpotent elements 
Xm £ six), 0 < m < g with two Jordan blocks of sizes n — m and m. Denote by q := ^k. 
Let Ai, A'^,..., Ag, Ag be Frobenius eigenvalues on H^{Gt /ordered so that AjA' = q. 
Also write Aq = 1, Ag = q. Let I = {1, .■.,g} if n = 2^ -|- 1 and I = {0, 1, ..., < 7 } if n = 2g + 2. 
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Write 

a,„{T) := (-1)” • E ( HP* + ) ■ 

S'C/,|S|=m Vies / 

Theorem 1.1. (Theorem \4-l\ and \4-li\ ) For 0 < m < g, we have (T) = ±am{T), 

where Tq{T) is the sum ofT(y{T') for T' running over different orbits in the stable orbit 
off. 

See the theorems for the sign and see Appendix[^for the normalization. When m = 0, Nq 
is a regular orbit and ao{T) = 1 which is well-known. When m = 1 it’s a subregular orbit, 
and the result was proven by Hales [ 3 . He also gave parallel results for other classical 
groups. Our result probably brings the suggestion that general Shalika germs, after all, 
could have reasonably nice closed formulas. 

The starting point of the proof of Theorem 11.11 is to find a particular sequence of test 
functions for dni) supported on 0 (T)x,_i/ 2 - These functions are made available by the 
homogeneity result of DeBacker (special case by Waldspurger) [3]. The description of 
these functions will be given in the beginning of Section [H For these test functions, the 
LHS of (II. 1|] counts /c-points on a sequence of specific varieties. It turns out that the 
theory of pencils of quadrics by X. Wang [22] can be used to relate these varieties to 
Sym"*(Cr), the m-th symmetric power of the hyperelliptic curve Ct- They then in terms 
give the numbers am(T) above. This is the main concept in Section[3l Briefly speaking, the 
phenomenon is that in Goresky-Kottwitz-MacPherson [6] we know certain orbital integrals 
can be understood as counting points on Hessenberg varieties (their definition of Hessenberg 
varieties are more general than others’, see [6l 1.5]). The situation for obtaining Shalika 
germs could be slightly more involved and we end up with counting points on quasi-finite 
covers of Hessenberg varieties (see m)- Our quasi-finite covers of Hessenberg varieties 
then happens to be strongly related to varieties considered by Wang. 

Section |4] contains most of the computation. We begin with the case of odd ramified 
unitary groups. In subsection 14.11 we read out the varieties that appear in the LHS of 
(II. ip for our test functions and apply the geometric result in Section [3l Next in subsection 
14.21 we use Ranga Rao’s method to compute nilpotent orbital integrals. With our simple¬ 
looking test functions thanks to homogeneity result of DeBacker, our computation reduces 
to a combinatorial sum over the Weyl group of G(0). In subsection 14.31 we state results 
regarding Shalika germs (instead of stable Shalika germs), as well as the results for even 
quasi-split ramified unitary groups. 

A consequence of Theorem 11.11 is result regarding stable distributions supported on the 
nilpotent cone (i.e. linear combination of nilpotent orbital integrals) and endoscopic transfer 
of nilpotent orbital integrals of ramified quasi-split unitary groups. This is the main content 
of Section [5j The basic idea is that Shalika germs are the coefficients comparing regular 
semisimple orbital integrals and nilpotent orbital integrals. Once we know these coefficients, 
we are able to derive, from the very definition of stability and endoscopic transfer of regular 
semisimple orbital integrals, corresponding results of nilpotent ones. 

The relevant elliptic endoscopic data are Un.^{E/F) x Un 2 {E/F) with ni -|- n 2 = n as 
endoscopy groups of Un{E/E). Assuming some conjectures of Assem (Conjecture [5T] and 
[53D, our result for nilpotent orbits with two Jordan blocks agrees with previous results of 
Waldspurger [2T] for unramified unitary groups. This also provides another evidence for 
Assem’s conjectures. In fact, it was this connection to endoscopic transfer which led us into 
believing the formula in Theorem 11.11 in the first place (see Remark l5.11|] . 
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In addition, in section [6] we describe how those Shalika germs we compute show up in the 
Harish-Chandra-Howe local character expansions for some supercuspidal representations. 
Since Moeglin and Waldspurger [12] showed that the coefficients in the load character expan¬ 
sions are related to the dimension of certain degenerate Whittaker models, we can produce 
examples where the dimension of degenerate Whittaker model are given by counting points 
on some “non-elementary” varieties. 


Acknowledgments. It’s the very pleasure of the author to thank his advisor Benedict 
Gross for his suggestion on studying this problem and for his stimulating ideas and guidance. 
He will also like to express his gratitude to Xiaoheng Jerry Wang, for introducing him to the 
theory of pencils of quadrics and its applications. Meanwhile he would like to thank Zhiwei 
Yun, for teaching and sharing with him many brilliant ideas related to Hessenberg varieties. 
He has also learned a lot from Thomas Hales about many fundamental ideas on Shalika 
germs, for which he deeply appreciate. Lastly, he would like to express his gratitude to 
Sam Altschul, Stephen DeBacker, Jessica Fintzen, Bao Le Hung, Fiona Murnaghan, Loren 
Spice and Jack Thorne for numerous inspiring and helpful discussions. 


2. Notations and setup 

We collect the notations. We have a non-archimedean local field F, its residue field k, a 
ramified quadratic extension E/F, and a fixed uniformizer vr € F such that G E. Also 
we denote q := ^k. We have G = Un{E/E) is a quasi-split unitary group that splits over E 
(such group is unique). We write g = Lie G. When orbital integral on G or g is concerned, 
we always identify an element with its G(F)-orbit. The assumptions char(A:) ^ 2 and either 
char(F) = 0 or char(F) > n are imposed. In fact we’ll mostly work with the assumption 
char (A:) ;§> 0, and leave it to Appendix [C] to explain how we can reduce the assumption on 
characteristic to those stated above. 

Write G = GLn/k- It has a standard representation V, which we equipped with a non¬ 
degenerate quadratic form (•, •). We define an involution 0 on G such that 0(h) = (h*)~^ for 
h £ G, where L* is the transpose of h with respect to (•,•). This induces an involution on 
g := Lie G which we also denote by 9. Let G(0) = (G®)° = SO^, g(0) = g^^^ = Lie G(0) 
and g(I) = g^^~^ the invariant and anti-invariant subspace of 6. There is a vertex x on the 
building such that G is the reductive quotient of G/g at x and G(0) the reductive quotient 
of G/f- We fix such a vertex x. Also see subsection 12.11 below for a more elementary 
description of G, G and x. 

Let g'’® C g be the subset of regular semisimple elements and g(l)'’^ = g^^ H g(l). For 
any T G g(l)'’®, the monic characteristic polynomial of T is denoted pt(x), and Gt = (y^ = 
Pt(x)) is the smooth completion of the hyperelliptic curve defined by pt(x)- 

Whenever we have a group variety H acting on a space X over some field AT, by an orbit 
(or the orbit of x G X{K)) in X(K) we mean a subset of X(K) of the form {h.x \ h G G{K)}, 
and by a stable orbit (or the stable orbit of x) we mean a subset of X(K) of the form 
{h.x \ h G G(iL*®^)} n X(K). The (stable) orbits discussed in this article will be either 
(stable) orbits in g(F) under the adjoint action of G, or (stable) orbits in g(l)(/i:) under the 
conjugacy action of G(0). 

The methods for odd ramified unitary groups (n = 2g + l) and even (quasi-split) ramified 
unitary group (n = 2g + 2 ) are largely the same, but most of the computation has to be 
carried out separately. In most of this article we only treat the odd case in detail, but 
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describe geometric tools needed for even unitary groups and list the results. In particular 
we will go with G = U 2 g+i{E/F) unless otherwise stated, and notationally reserve n for 
other variables. 

2.1. An elementary description. We give a down-to-earth description of groups G, 
G, the involution 0 and the vertex x. Let V be an n-dimensional hermitian space over 
E, spanned by basis vectors ci, ..., and equipped with the hermitian form given by 
= Ya=i <^n+i-ib*, where ai,bi E E and b* is the conjugate of 6 * over 
E. Then G is such an algebraic group defined over F for which G{F) is isomorphic to the 
group of unitary operators on V, i.e. i?-linear operators on V preserving the hermitian 
form. 

Let A = span(p^{ei, ...,e„} be a lattice in V. Let K be the subgroup of G{F) consisting 
of unitary operators g with g{A.) = A. Then K stabilizes a unique vertex on the Bruhat-Tits 
building of G over F, which (up to conjugation) is the vertex that we call x. We have the 
stabilizer group G(F)x = K. 

The hermitian form (•, ■)herm takes Oe values on A. Its reduction mod thus defines 
a quadratic form (•,•) on V := A/tt^/^A. Write ei,...,e„, to be the reduction of ei,...,en, 
respectively. Then (•, •) on V is dehned by (J2 bi^i) = o-n+i-ih, where ai, bi E 

k. The algebraic group G then should be identified with the group of automorphisms of 
V (not necessarily fixing (•,•)); G{k') = GL{V <Sik k') for any finite extension k'/k, and 
6 r\ G{k') is the involution 9{g) = {g^)~^ where g^ denotes the transpose of g with respect 
to the quadratic form (•,•). 

The Lie algebra q{F) is the space of anti-hermitian endomorphisms of V , and for any 
d E iZ, g(F)x,rf = {A E 0 (T) | A(A) C vr-^A}. We have ~9{FUd/UFUd+i/2 = Q^ik) 
given by hrst scaling 7 r“'^ and then modulo Of course, this map depends on the choice 
of uniformizer E E. Here recall g{2d) = if d is integral and g(2d) = g®^“^ = 
Syrin?{V) if d is half-integral but non-integral. 

The algebraic group G has 0-stable Borel subgroups. For example, one such B is given 
by that B{k) consists of endomorphisms of V that sends Cj to a linear combination of ei, 
62 , ..., and Cj. We also denote B(0) = B (1 G(0) = {B^)° . They are used in Section HI 


3. Geometric result via pencils of quadrics 
In this section, k can be any perfect field with char(fe) ^ 2. 

3.1. Odd case. In this subsection we have n = 2g-\-l and G = GL2g+i/fc = GL(H). Recall 
that the vector space V comes with a non-degenerate quadratic form (•,•). We then have 
in the introduction an involution 0 on G which sends g to ( 5 *)”^, where g^ is the adjoint 
of g with respect to (•,•). This induces an involution on g, and we write g(0) = g^^^, 
g(l) = We have g(l) = Sym^(F) as G(0)-representations. As (•,•) provides a self¬ 

dual structure on V, g(I) = End'^®^'^““'^^(I/) is also the space of self-adjoint operators on 

R. 


The representation G(0) r\ g(l), or equivalently SO(I/) r\ Sym^(I/), was considered by 
Bhargava-Gross in [2]. An orbit in this representation is GIT-stable iff it’s contained in 
g(l)^® := g^'^ n g(l) where g*”^ is the open subset of regular semisimple elements in the Lie 
algebra. We now fix an T E g(l)'’^(A;). 
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Let pt{x) be the degree 2g + l monic characteristic polynomial of T. Let L = k[x\/pT{x) 
be a degree 2g + 1 etale algebra over k. Consider the Weil restriction Res^/r 2 - This is 
a commutative etale finite group scheme over k of order 2^®+^. It has a surjective norm 
map Nm : Res^/i 2 P 2 - Bhargava and Gross observed for T G 0(1)^^^, we have canonical 

isomorphism StabG'(o)(T) = ker(Res^^^^^^^^^V 2 P 2 )- In fact, the map T i-A pt{x) is the 

GIT-quotient map 0 ( 1 ) i-)- 0 ( 1 )//G(O); we have 0 ( 1 )//G(O) = is the space of degree 

n monic polynomials. 

Let Ct = = pt{x)) be a (smooth completion of) genus g hyperelliptic curve. Let 

Jt = Pic'^(C''r). Since the 2-torsion Jr[2] is generated by differences of Weierstrass points, 

one checks Jt[ 2] = ker(Res^^^^^^^^*^/r 2 P- 2 )- Consequently Jt[ 2] = StabG'(o)(T). 

If one fix such a T, then the orbit of T is G{0){k).T while the stable orbit of T is 
(G(O)(fc).T)n 0 (l)(fe). There could be more than one orbits inside a stable orbit, and relative 
to the choice of T as a pinning they can be classified by H^{k, Stabc(o)(^)) H^{k, G( 0 )). 
When A: is a finite field, by Lang’s theorem, the latter pointed set is trivial, and thus we 
have LI^(/c, Stabc(o) (^)) — Jt[2]) classifies orbits in the stable orbit of T relative to 

the choice of a pinning. 

The GIT-quotient map 0 ( 1 ) —>• 0 ( 1 )//G(O) has a Kostant section [ini Thm 5.5]. Using 
the Kostant section as a pinning, a G(0)(/i:)-orbit in 0 (l)'’^(/c) corresponds to a hyperelliptic 
curve Ct together with a class in H^{k, Jt[2]). For k a global field, Bhargava, Gross and 
others used this to study the average size of 2-Selmer groups of such hyperelliptic curves, 
see e.g. [3]. For this purpose, X. Wang developed the theory of pencil of quadrics [22] . 

It turns out that his theory is very useful in describing the variety that we’ll encounter in 
orbital integrals. 

Define on U©A: two quadratic forms by ((ui. Cl), (u 2 , C 2 ))i = (ui, U 2 ) and ((ui, ci), (u 2 , C 2))2 = 
(ui, ru 2 ) — C 1 C 2 . This defines a generic pencil of quadrics in the sense of X. Wang [22l Intro.]. 
Recall that a subspace W C U © /c is said to be isotropic with respect to a quadric (e.g. 
(•,-)i) if the restriction of the quadratic form to W is trivial. In his paper, Wang proved 
the following: 

Theorem 3.1. (Wang [221 Thm. 2.26]) Let Ft be the variety that parametrizes g-dimensional 
subspaces of V (B k that the are isotropic with respect to both (•,-)i and (•,-) 2 - Then there 
is a commutative algebraic group structure on 

Gt ■= Jt LI Ft U Pic^{CT) LI 


where Ff, = Ft as a variety, the addition law on Jt U Pic^{CT) agrees with that of 
Pic(C'r)/(2(oo) = 0), and Gt has component group equal to Z/4. 


In particular, Ft is a torsor under Jt and there is a doubling map x 2 : Ft —?■ Pic^(C'r). 
We review the group structure in the theorem. The group structure is determined by 
(p) — [W], i.e. how to subtract from p G Ct a subspace \W] G Ft- This is done as follows: 
a point p = {x,y) on Ct corresponds to a ruling of (•, •)2 — x{-, •)!. Recall that a ruling 
is a connected component of the variety parameterizing [g + l)-dimensional subspace on 
which the quadratic form is trivial. There will be a unique {g + l)-dimensional space W' 
in the ruling such that W' D W. Inside the space W' there will be, when counted with 
multiplicity, two y-dimensional subspaces W and W" on which (•, •)! and (•, •)2 vanish. It 
is then defined (p) — [IF] := [IF''], and this uniquely characterizes the group structure on 
Gt- 
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It’s obvious that Gt depends only on T up to G(0)(/c)-conjugacy. As mentioned in the 
introduction the orbit of T in its stable orbit may be characterized by a class in H^{k, J'r[2]). 
This class can be describe as follows: the map x 2 : Ft —>■ Pic^(C'T) is etale and Galois 
with Galois group being Jr[2] as a group scheme over k. There is a distinguished rational 
Weierstrass point oo € Ct C Pic^(C''r). Then the class is the torsor (x2)“^(oo). 

For any 0 < m < 5 , consider jm : Sym^(C t) Pic^(Gr) hy jmipi, ■■■,Pm) = ipi)+ ■■■ + 
(Pm) — {m — l)(oo). Let Xx^m be the image of jm, and let XT,m ■= {XT,m) be its 

preimage under the etale map x2. We also take Xt,-i = 0- We shall relate Xx^m with 
the following varieties Fx^m, which could be thought as a generalized version of Hessenberg 
varieties considered by Goresky, Kottwitz and MacPherson [ 6 ]. 

For any finite extension k'/k, we call a flag of feLsubspaces 0 C C ... C C 
{V © k) k' good if 

(i) dim VF* = i. 

(ii) The restriction of (•, •)! and (•, •)2 to is zero. 

(hi) ^V®kk'. 

(iv) T{W^) C VI < i < 5 - 3. 

(v) r(W3-2) c 7 ri(W 3 ). 

Here tti : (P ® k) k' ^ V k' is the projection to the first factor. For 1 < m < g, 
a good flag is called m-good if T{W^~^) C (where = {0} is understood). 

Also a good flag is called 0-good if C V ©^ k'. Next, for 0 < m < 5 , an m-good flag 
is called m-excellent if it is also n-good for m < n < g. On the other hand, a good flag 
is called m-general if it is not n-good for any 0 < n < m. Finally, a good flag is called 
m-exact if it is m-excellent and m-general. Now let 

FT,m{k') = {0 C C ... C C (y (Bk) ©^ k' \ This is an m-exact flag}. 

The functor Ft,™ is easily seen from its very definition to be represented by a quasi- 
projective variety over k which we’ll denote with the same notation. In fact, there is 
a projective variety FT^good that parameterize good flags, and Fx^m C Ft, good is locally 
closed. There is a natural map j : Ft, good Ft by sending a flag to [VF^]. This section is 
devoted to the proof of the following result; 

Theorem 3.2. For 0 < m < g, the map Jlrrm '■ FT,m Ft is a locally closed embedding, 
with image equal to XT,m \ XT,m-i ■ 

A more direct proof of this theorem in the case m < 2 was shown to me by X. Wang. 
Nevertheless, we begin our proof for the general case with two simple lemmas: 

Lemma 3.3. IfOCWCV is such that (•, •) is trivial on W, then r(lF) / IF. 

Proof. Let IF-*- = {v ^V\{v,w) = 0, Vu) G IF}. Then IF C IF-*- by assumption. Suppose 
on the contrary r(lF) = VF, then by adjointness r(lF-*-) = IF-*-, and r|vF is the adjoint of 
T\v/wr- But this says T\w and Fly/vpi have the same eigenvalues. Hence T cannot be 
regular semisimple. □ 

In the rest of this section, we work “geometrically,” i.e. we replace k by an algebraic 
closure k, so that we can omit the notations • ©^ k' and so on. This will make no harm to 
what we want to prove. 

Lemma 3.4. Let 0 < m < g. 
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(i) Suppose 0 C C ... C C V ® k and 0 C {W^)' C ... C (W^)' C V ® k are such 
that = iW^y. If one of the flags is m-general, then W'' = (VF*)' for g — m < i < g. In 
particular the other is also m-general. 

(a) // 0 C C ... C cV(Bkis m-good, then there is a unique 0 C {W^)' C ... C 
{W^y <ZV ®k which is m-excellent such that PF* = (VF*)' for g — m < i < g. 

(Hi) If in (i) both flags are m-exact, then the two flags are the same. 

Proof. If m > 1, then n F is unique. Next if m > 2, then the flag is not 

1-good and T{W^~^) (f. W^. Since goodness requires r(VF^“^) C VF^, we have IF^“^ = 
\Y9-^ n T~^(W^) is also unique. Proceed similarly and we have the uniqueness of IF^“^, 
..., IF^”™-. This proves (i). Now suppose the flag is m-good and (m -|- l)-good. Then 
T(VF^“™') C VF5“™-+i and T(VF^“™'“^) C VF^“™-. However the previous lemma implies 
T{W9-^) VF5-rn_ Thus IF5-»«-i = yi/g-m p jg possibility for this 

to hold, i.e. for the flag to be (m -|- l)-good. Continue the argument and we obtain (ii), 
and (iii) follows immediately. □ 

Let C Ft, good be projective varieties parameterizing m-excellent flags. The key is 
Lemma 3.5. j{FfI,^) C XT,m- Also when m = 0, j{Ff?fl) = X-Tfl. 

Lemma 3.6. Lemmxi \S. ,51 above implies Theorem AS.^ 

Proof. The second statement in Lemma 13.51 gives the theorem when m = 0. We now use 
induction on m. Let FT,m C Ffl^^ be the open subvariety that parameterize those flags that 
are m-excellent and (m — l)-general. We have FT,m-i C Tt,™ as a closed subvariety. By 
induction j gives an isomorphism F^m-i — XT,m-i \ XT,m- 2 , which is (m — l)-dimensional. 
In particular FT,m-i C FT,m are both non-empty. 

On the other hand, a dimension count shows that Ft, good has dimension at least g, and 
C Ft, good is a closed subvariety cut out by ^ — m equations. As FT,m is open in 
every component of has dimension at least m. This says that Fr,m = \ F^m-i 

is non-empty (as a variety). Lemma 13.51 will force the image of F^m under j to be inside 
the m-dimensional locus XT,m,, and Lemma l3.4l iiii says that the dimension of the image 
has to be the same as the domain. Since dim = rn, we have j{FT,m) C is dense. 

Since Ffl^ is proper, j{Ff?)^) = XT,m- We also have j{FfI^_fl) = Ar,m-i by induction. 
By Lemma [T^ iii. the image of FfI^\FT,m under j is in XT,m-i- By Lemma iT^ ii . the 
image of F^m is disjoint from A^m-i- Thus j{FT,m) = XT,m\XT,m-i- The proof of the 
uniqueness in Lemma I3.4lf ii can be carefully checked to imply that not only j is injective 
on closed point, but also j : FT,m — XT,m \ XT,m-i is an isomorphism. □ 

Proof of Lemma \3.5[ The case m = 0, including the second statement, is precisely [221 
Remark 2.30]. We review it here. 

The variety A^o is by definition (x2)“^(oo). Let r(oo) : V Q) k ^ V (B k he the map 
sending (u,c) to (v,—c). Then for any [PF] G Ft, one checks from the definition that 
[r(PF)] = (oo) — [PF]. The variety (x2)“^(oo) thus parameterizes g'-dimensional varieties 
PF^ in F © /c, stabilized by r(oo), on which (•, •)! and (•, •)2 vanish. 

Since (•, •)2 does not vanish on the second factor of F ® fc, this forces PF^ to lie com¬ 
pletely in the first factor, i.e. (x2)“^(oo) parameterizes ^-dimensional varieties PF^ in F on 
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which (•,-)i|\/ = (•,•) and (•,-) 2 |y = vanish. Lemma [33)^ ii),(hi) then says it extends 

uniquely to a 0-exact flag (there we began with a good flag that is 0-good instead of only 
W^, but the same proof also applies to the present case). This proves the m = 0 statement. 

The JrpJ-structure on (x2)“^(oo) can be described as follows: Let po, ...,p 2 g,oo be the 
Weierstrass points of Ct- Then Jt[2] is generated by {{pi) — (oo)), 0 < i < 2g with the 
only relation that YliLoiiPi) ~ (oo)) = 0. The action of (pi) — (oo) is described as follows. 
Say Pi = (x,0). Then x{-, •)! — (•, •)2 is a degenerate quadric with 1-dimensional kernel U. 
For [W] G (x2)“^(oo), x{-, •)! — (•, •)2 is trivial on the {g + l)-dimensional space W + U. 
There will be exactly one ^-dimensional subspace W <Z W + U, other than IF, on which 
(•, •) and (-jT-) are also trivial. It is then defined {{pi) — (oo)).[IF] = [IF']. 

We now discuss the general case. From now on 0 < m < is hxed. We shall show 
^ ^T,m for a generic T G g(l)'’'' (i.e. for T in a Zariski open subset of g(l)'’^). In 
fact, what we will do is the following: Fix a flag F = (0 C C ... C IF^“^ C IF^ C F© A;) 
such that C V, V, k (here V and k are the hrst and the second 

components in F © k), and (•,•) is trivial on 7ri(IF^). Since all such flags in F © /c are 
conjugate by G(0) (where G(0) preserves F C F © A: and acts trivially on the second 
component), without loss of generality we may assume that F is exactly the flag in interest. 

There is an irreducible closed subvariety V C g(l) such that the flag is m-excellent with 
respect to T G g(l)'^^ if and only if T lies inside V. There is an Zariski open subset of V 
consisting of those T for which the flag is m-exact. What we shall prove is that for an even 
smaller open subset A C V, all T G A satisfy j{¥) G XT,m- A continuity argument by 
having Theorem 13.11 in family then extends the result to all T G V, which is what we need. 

The case m = g is trivial, and we’ll assume 0 < m < g. Let 0 C IF^ C ... C VF^ C F © A: 
be an m-good flag. By assumption C Consider Uq = VF^”™ and = 

:= {(u,c) G F© A:| (u,u;) = 0,Vui G (that is, is used to denote 

the orthogonal complement with respect to (•, •)!). Following the spirit of [221 Sec. 3.1], we 
define inductively subspaces Uq <Z Ui <Z ... C ?7|^m+ij C c c ... C as 

follows: 

(Un := ([/»^-i)-Li nIF^'-™+2’"-b 

[u^ := {Un)^^nU^-\ 

Lemma 3.7. For 0 < n < we have 

(i) Z) ('p|/9-m+2n+l^±l_ 

For 0 < n < we have 

(a) dimUn=g — m + n. 

(in) dim = g + m — n + 1. 

Proof. For n = 0 it’s obvious. We now do induction on n. For (i) let 0 < n < J. Since 
Un C C IF^“^ C F, we have F"' D k. Now F is good (with respect to T) says 

T(IF^'-’”+2»^-1) C 7ri(IF»-™+2"+i). Since Un C By definition of (•, •)2 we have 

(Fn)-L2 D AlsO by induction F**-! D (^ 9 -»n+ 2 n-l)±l 3 ('p^g-m+ 2 n+l)±l_ 

This gives (i). 

For (ii), since F'^"^ D (^ 9 -»n+ 2 n-i)±i i^y ^ C + k, 

where k denotes the second component in F © A;, i.e. the kernel of (•,-)i. Since k 
y^g-m+ 2 n-i^ this says dimF„ = dim(F"'“^)-'“^ — l = g' — m + n. This proves (ii). 
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Lastly for (ill), by definition (It’s important for our construction 

that it holds when n = 1!). Since Un contains Un-i with codimension 1 by (ii), it suffices 
to show ^ U^~^. Suppose on the contrary (Un)^^ 3 U^~^, then we have U"‘~^ C 

n In this case we have k C Un C V = From the definition of 

(•, •)! and (•, •)2 we get 7ri(C/”“^) C (Un)~‘~nT(Un)~‘~, where this time means the orthogonal 
complement in V with respect to (•,•). But this is impossible, because by Lemma 13.31 
Ll r(C/„)-’- intersect non-trivially and thus have dimension less than g — m + n by (ii), 
while dim7ri([/”“^) = dim 1/”“^ — l = 5 r + m — n + lby inductive hypothesis from (hi). □ 


We now come back to the proof of Lemma[331 Define L to be the variety that parametrize 
(^-dimensional subspaces W satisfying C/|^m±ij C VF C I/L 2 J and that (•,-)i and (•,-)2 

vanish on W. In particular by construction we have I/im+i 1 C l-i and 

L 2 J 

W9 C (IF»)^2 n C i.e. [W^] G L. 

Define V to be the subquotient V := /U^ m+i Since f7|^m+ij C ([/l-“5^-l“^)-*-^ c 

(17l-“^J)-*-^ and C ^ f7|^m+ij C (C/L^^J)-*-^, the two quadratic forms 

(•, •)! and (•, •)2 restricts to be quadratic forms on V. Denote still their restrictions by (•, •)! 
and (•, •) 2 , respecitvely. Let L' be the variety that parameterize [yj-dimensional subspaces 
in V on which (•, •)! and (•, •)2 are trivial. Then evidently L = L'. 

Consider the polynomials p^^\x) = disc ((•, •)! — x{-, •) 2 ) \ui/Ui * = 0; 1) • We 

claim p^\x) = ^ ^^ 2 ;). To prove this, observe that when we go from U^/Uq 

to U^/Ui, we quotient out Ui/Uq, on which both (•,-)i and (•,-)2 is zero. Even more, 
Ui/Uq is in the kernel of (•,-)i while {Ui/Uq, /U ^)2 is non-trivial. This exactly says 
p'^\x) = x‘^p^\x). Repeating the argument gives the asserted result. 

L' comes from the situation in |22l Sec. 2.1], namely that of a generic pencil of odd 
dimension. As explained in the beginning of this proof, |221 Remark 2.30] says that L' is a 
torsor under the 2-torsions of the Jacobian of a hyperelliptic curve of genus [^J • And this 

hyperelliptic curve is simply Ct = {y^ — Vt ^ ^(^))‘ 

Recall T G V is such that our flag F is m-excellent with respect to T. One checks from 
definition that when T runs over the subspace V, p^^ (x) runs over all polynomials of degree 
less than or equal to 2m +1 that are divided by Consequently, there exists a Zariski 

(I ’TI+l J) 

open subset A C V such that for T G A, ^ (x) is separable (i.e. having distinct 

roots). 

For such T, let po, ...,p 2 [^J j 00 be the Weierstrass points of this hyperelliptic curve. We 
have ((po) — (oo))-|-...-|-((p 2 [i!ij) — ( 00 )) = 0 G Pic°(C''r). By the Pic°(C''r)[2]-torsor structure 
on L' as described in the m = 0 case at the beginning of this proof, we have a sequence Dq, 
Di, ..., D 2 [^J) ^ 2 [^J+i = ^0 such that fli and Dj+i intersect in codimension 1. When m 
is even, pt(x) is divisible by x, which means that one of the Weierstrass point pi = (0, 0), 
i.e. the quadric (•, •)! is degenerate on V, and (•, •)! vanish on the sum Dj J- Dj+i for some 
i. 


Now recall that R is a subquotient of R 0 /c. The preimage of Dq, Di, ..., 

D 2 [^j+i = Do is a sequence Qq, ..., D 2 [^j+i of subspaces in R 0 A: with Dq = D 2 [^j+i = 
IR^, and that Dj and Dj+i intersects in codimension 1. This says that there are points 
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Pi, •■••,P2Lf J+1 on Ct such that ((pi)-((p2)-----((P2Lf J+i)-[^^])-")) = and one of 
Pi is oo when m is even. By definition of the group structure, this says (x2)([fB®]) G XT,m- 
We have therefore hnished the proof of the lemma and hence that of Theorem 13.21 □ 


Remark 3.8. I hrst learned from Jack Thorne the idea that symmetric powers of Ct should 
arise, which he observed in his unpublished work generalizing his results in [18] to nilpotent 
orbits of two Jordan blocks in type A. In fact, if one substitutes this whole section with 
corresponding result in |18[ Thm 3.7] and re-apply the method in Section (H one can obtain 
stable Shalika germs for subregular nilpotent orbits in terms of number of rational points 
on the curves in [181 Thm 3.7]. This shows, for example, that certain stable subregular 
Shalika germs of Eq (resp. Ey, Eg) will be given by counting points on non-hyperelliptic 
curves of genus 3 (resp. 3, 4). 


3.2. Even case. In this subsection n = 2^ + 2; G = U 2 g+ 2 {E/E), G = GL2g+2/k = GL(1/), 
G(0) = SO(l/) and 0(1) = Sym^(E) where E is a 2(7 -|- 2 dimensional non-degenerate split 
quadratic space. The method in this subsection is almost identical to that of the previous 
one, and we only list the setting, definition and results here. 

We have parallel result to Theorem 13.21 Again fix T G 0(1)'’^(A;). We also write pt{x) S 
k[x] the monic characteristic polynomial of T. Our hyperelliptic curve Ct '■= {lP = Pt{x)) 
now has two points above the infinity on We shall denote these two points by and 
They are both defined over k. 

Consider L = k[x\/pT{x). The Weil restriction Res^/i2 now has not only a surjective 
norm map Nm : Res^//2 —t P 2 but also a diagonal embedding A : 712 —t Res^/i2- We 

have Stabo(y)(T) = Res^/i2, also StabG'(o)(^) — ker(Res^/X2 P 2 ), and lastly Jr[2] = 

StabG(o)(r)/Z(G(0)) ^ (ker(Res^7^2 ^ P 2 )) /X{p2). 

We denote by = (•,•) the standard quadratic form on V, i.e. the one which is 

invariant by G(0). Then and are just the two rulings of (•, •)!. Define (•, •)2 on 
V by {vi,V 2)2 = {vi,Tv 2 )i- Then the theory of pencil of quadrics says: 

Theorem 3.9. (Wang [22( Thm. 2.26]) Let Ft be the variety that parameterizes g- 
dimensional subspaces of V that are isotropic with respect to and {■,-) 2 - Then there 

is a commutative algebraic group structure on 

Gt := Jt U Ft U Pic^{CT) U F)., 

where F^ = Ft as a variety, the addition law on Jt U Pic^[CT) agrees with that of 
Fic(C'r)/((oo(^)) -|- (oo^^^) = 0), and Gt has component group equal to Tij^. 

We again write the doubling map x2 : Ft —)• Pic^(Cr) which is etale Galois with Galois 
group Jt[2]. For 6 <m<g with m even, define jm : Sym™'(C'r) —Pic^(CT) by 

j^m\pi, ■■■,Pm) = (Pl) + ••• + (Pm) “ “ l) 

jinHPl, ■■■,Pm) = [pi] + - + {Pm) “ y(oo(^)) “ “ l) (oo^^)). 

And we dehne to be the image of jm , and Xt^^ ~ (^2)“^ ^ = 1; 2. 
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For 0 < m < g with m odd, we define jm ,jm\ jin ■ Sym^{C t) —> Pic^(C'T) by 


jinHPU- 

■,Pm) = {pi) + ■■ 

■■ + {Pm) 

^ ^ f2'l\ 

2 2 

JmiPl,- 

■,Pm) = {Pl) + 

•• + (Pm) 

m —3, Q-i, m-|-1 , (^ 2 ^ 

2 2 

jinHPl,- 

■,Pm) = {pi) + 

■■ + (Pm) 

m-kl Q') m-3 ( 2 )s 

-^(oo^b)-^(oo^b). 

And we define Xf', 

^ to be the image of jm 

,andxi^l = {x2)-^ , i 


Next, we introduce the notion of good flags. A flag of subspaces 0 C C ... C C 

V is called good if 

(i) dim VF* = i. 

(ii) The restriction of (•, •)! to jg zero. 

(iii) The restriction of (•, •)2 to is zero. 

(iv) T(W^) C yi<i<g-l. 

For 0 < m < g, a good flag is called m-good if T(1T®“™') C Here W~^ = 

= {0}, i.e. good flags are automatically g'-good. A flag is called m-excellent if it is 
n-good for m < n < g. 

For 0 < m < g, a good flag is called m-general if it is not n-mod for any 0 < re < m. 
For any 0 < rre < 5 , we now define the notion of rre-exact flagu. Let be any 

m-excellent and m-general flag. There always exists another m-excellent flag 
satisfying but U9+^ / if m is odd, or = W9-\ but 

jjs ^ if m is even. We say is m-exact if is also m-general. Lastly, a 

0 -excellent flag is said to be 0 -exact. 

Now let be the variety that parameterize m-exact flags for which is in the 

ruling 00 ^^^, and F^j^ be the variety that parameterizes those m-exact flags for which 
is in the other ruling 00 ^^^. We have natural maps j : FT^m '■= Fi^l^UFi^]^ —>■ Ft by sending 
to W9. 

Theorem 3.10. For 0 < m < g, the restriction of j to F^j^ is a locally closed embedding, 
with image equal to 

where for F^j^, we replace, in the case m is even, the two superscripts by . 


Remark 3.11. If we relax the condition of m-exactness to require only to be m- 

excellent and m-general, then the image of j\ (p will be in the even case and 

™ case. However our definition of m-exactness is what one should 

use for orbital integrals on even ramified unitary groups in Section [H 


^See also Remark l3.11l 
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4. Main computation 

In this section we have n = 2g + 1 except for a part of subsection 14.31 where we’ll state 
differently. We work with the assumption that char(/i;) S> 0, and leave it to Appendix ICl to 
explain why this assumption may be dropped. 

The nilpotent orbits O € O{0) of q{F) are classified as follows: The stable orbits, just 
like in 0 l 2 g+i) ^-^e classified by partitions A = (A“^...A“'’) of 2 g' + 1 which give the sizes of 
the Jordan blocks, that is Ai > ... > A* and = 2g + 1- In such a stable orbit, the 

orbits are classified by 

n d, = i-irNE/^EX}, 

Xi odd 

in which di G /N^jpE^ (= ^ 2 ) if cti{\i — 1 ) is even and di G tt^^‘^{F^/N^ ipE^) 
(a torsor of ^ 2 ) if Oii{\i — 1) is odd. We’ll also denote by (A, the corresponding 

nilpotent orbit in g. 

This classification goes as follows: let V be the standard representation of G/p, i-e. V is 
a {2g + l)-dimensional hermitian space over E, with hermitian form (•,•). Write G g an 
arbitrary element in such a nilpotent orbit with a* Jordan blocks of sizes Aj. There exists 
a unique decomposition V = 0*=^ Vi such that N preserves each V, that all Jordan blocks 
of N\y, are of size A^, and that different Vi and Vj are orthogonal under (•,•). 

We have by definition induces an isomorphism from Vi/N{Vi) to ker(iV|p,). Also 

one has by the anti-hermitian property of N that {N{Vi),keT{N\y )) = 0. This allows us 

to consider a pairing on Vi/NiVi) by This pairing is non-degenerate, and it 

is hermitian if Aj is odd and anti-hermitian if Aj is even. The invariant di is then the 
discriminant of this pairing. 

Similarly, we can speak of nilpotent orbits in g(l)(A:), i.e. G(0)(A:)-orbit in g(l)(A:) that 
are nilpotent in g. The stable orbits correspond to the same partitions, and the orbits 
inside a stable orbit are classified by 

The classification is done like above by replacing V hy V (the standard representa¬ 
tion of G), both hermitian and anti-hermitian forms by quadratic forms over k, and both 
E^/Np/pE^, /Np/pE^) by k^/k^'^. It’s not hard to check that this set is in 

bijection with the previous one. For our purpose we’d like to give a canonical bijection 
as follows: for any N G g(I)(A:) nilpotent, there exists a lift N G g(T)x,_i /2 which is also 
nilpotent. The orbit of such N is uniquely determined by the orbit of N. 

Let A^o G g(l)(A:) be an arbitrary regular nilpotent element, i.e. one with a single Jordan 
block. For I < m < 5 , let G g(I)(/c) be a nilpotent element with two Jordan blocks of 
sizes 2 g'-|-l —m and m whose orbit is classified as (( 2 ^- 1 -1 — m,m), (— 1 )^, l)g(i) if iR is even 
and {{2g -|- I — m,m), I, (—l)5)g(i) if m is odd. Write N^, for the corresponding nilpotent 
orbit in g(F’). For m > 0 it’s classified by {{2g -|- 1 — m,m), (—1)®,' 7 r“^/^)g(i) if m is even 
and {{2g -|- I — m, m), (—l)^)g(i) if m is odd. 

When m > 0 there are always two orbits in the stable orbit of Njn- We again fix 
T G g(I)^^(/i:) and a lift T G g(T)x,_i /2 in this section. We shall prove our main theorem 
(see Theorem o for the definition of am{T))\ 

Theorem 4.1. For 0 < m < g, we have F^ (T) = am{T) for any lift T of T. 
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Also r^, (r) = (—l)™'am(7") for the other nilpotent orbit N'^ in the same stable orbit. 


For notational convenience, in this section we only compute the Shalika germs for N^- 
For the other orbit the computation is identical except that we should replace Ct by its 
quadratic twist, resulting in the sign (—1)"* in the theorem. 

We want to plug in (|l.ip some test functions / that are locally constant by a “sufficiently 
large” lattice and for which we know how to compute J{T, /). Let S G B G G he a choice 
of 0-stable maximal /c-torus and Borel A:-subgroup (see also the end of subsection EH). Let 
b = Lie B and let B{0) = B G G(0), b(i) = b H g(i), so that b = b(0) © b(l). The same 
notations apply to S. 

There exists a point y on the Bruhat-Tits building, which can be taken to be the barycen- 
ter of some alcove neighboring to x, such that 0(T)y _i /2 is the preimage of b(l)(/c) under 
0 (T)x,_i /2 0(1)(^)- We make the following hypothesis, which holds by [U Thm. 2.1.5] 

when char(A:) is large enough (compared to g). 

Hypothesis 4.2. Equality li. 11) holds for any compactly supported function that are locally 
constant by 0(T)y _i/ 2 - 


Now we can choose our test functions. Fix now 0 < m < g and let Nm be as before. 
After a conjugation by some element in G{0){k), we may and shall assume that there exists 
a cocharacter pm ■ Gm/k <5(0) such that Pm{^) acts on Nm by A“^. Write Qj G 0 for the 
subspace on which Pm{^) acts by , 0(l)j = Qj H 0(1) and 0(l)>i = 0j>j0(l)j- We may 
and shall assume that pm lies in the correct Weyl chamber so that b = 0(l)>o. 


Let U be the unipotent radical of B. U{h) = U G B{{f) is the unipotent radical of B{0). 
Let fm G G^{q{F)) be the function with support inside q{F)^ _i /2 defined by fm{X) = 
= {ffU{0){k))~^ if the image of X in 0(1) is in the affine subspace A'm + 0(l)>-i, and 
fm{X) = 0 otherwise. Theorem 14.II follows evidently from Hypothesis 14.21 and the following 
two propositions. 


Proposition 4.3. For 0 < m < g, 


r\f,fm) = Y. ^rn'{T) 

m'=0 


+ ( g - m' \ 


E 


2 J ^ / _ / 

m—m'—j _ m—m'—j—l\ (9 ^ 


Here fm) is the sum of J(^^ fm) where T' runs over representatives of the orbits 

in the stable orbit of T. 


Proposition 4.4. For 0 < m' < m < g, 

/ " \ I 

J{Nm',fm) = 2 


^ f 9-m' 

,L^J 


^ ^qrn-m' 
1=0 


—m'—j _ „rn—m'—j—l\l 9 ^ 


For nilpotent orbits O other than (the orbit of) Nm' with 0 < m' < m, J{0,fm) = 0. 


4.1. Geometric identification. The goal in this subsection is to prove Proposition 14.31 
We begin with 
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Lemma 4.5. There is a natural bijection between G{0){k)-orbits of T in its stable orbit 
and G{F)-orbit of T in its stable orbit. 

Proof. For this proof only we’ll replace G by SU 2 g+i{E/F). One checks that this re¬ 
placement does not affect the orbits. The orbits in the stable orbit of T are classified by 
ker(i?^(A:, Stabc(o)(^)) and that of T by ker(if^(F, Stab(^(T)) —)■ H^{F,G). 

By Lang’s theorem and the fact that simply connected group over a non-archimedean local 
field has trivial we have H^{k,G{0)) = H^{F,G) = 0. Recall also that StabG'(o)(2^) — 
Jt[2]. 

The key is that our T has its centralizer Gf is anisotropic over the maximal un¬ 
ramified extension of F [ISl Thm 2.1], Consider the exact sequence 

1 ^ iLi(Gal(F“7F),Stabg(f)(F“^)) ^ iL^(F,Stabg(f)) ^ {F^\Siab^{f)) 

The last cohomology group is trivial by Steinberg’s theorem. The first cohomology group 
is isomorphic to H^{k, because Jr[2](fc) is a quotient of Stabg(T’)(F“’') with kernel 

possessing a filtration with graded pieces = Gq. 

This finishes the proof of the lemma. Note that from the exact sequence, one also sees 
that all orbits in the stable orbit of T appear in q{F)^ _i/ 2 , and the bijection just established 
is compatible with the reduction map 0 (T’)x,_i /2 ^ 0(l)(fc) that sends T i—)• T. □ 

The main result in this subsection is to translate from the subsection [3T] that 


Lemma 4.6. For 0 < m < g, 


J{T,fm) = 


#Jt[2](A:) 


iif^T,mik) - ffXT,m-lik)). 


and 


= ffXT,m{k) - ffXT,m-l{k). 


Proof. To ease notation we deal with the case m > 0. The proof applies to m = 0 case with 
a little change in various places. By [I9l Thm 2.1], for h G G{F), Ad(h)T € q{F)^ _i /2 iff 
h G G{F)x. In particular the centralizer Stabg(T)(F) C G{F)x. Moreover, G(F)x,i /2 acts 
trivially on fm since fm is locally constant by 0(F’)x,o C 0(T)y _i/ 2 - The integral is thus 
essentially a sum over G{F)x/G{F)^ ,1/2 = 02g+iik). 


The measure of G{F)^ 1/2 is equal to that of 0 (F’)xq/ 2 , which in Appendix lAl can be 

checked to be q~ ^2 ^, The measure of Stab( 5 (T)(F)i /2 = Stabg(T)(F) n G(F’)xq /2 is 
1. The image of Stab(^(T)(F) in 02g+i{k) is equal to Stabo 2 g+i(fc)(T), which has order 
2#Jr[2](/c). Also \D{T)\ = We thus have 


(4.1) 

J{f,fm) 


1 

2#Jr[2](A:) 


^ /^(Ad(h)(r)) 

h£02g + l{k) 


1 

#Jt[2](A;) 


UAdCh){T)), 

hGS02g + lik) 


where fm in the RHS is understood as a function on 0 (F’)x,_i/ 2 / 0 (F)x,o — 0(1)- 


We have pm acts on V (the standard representation of G(0) = S02g+i) by weights 2g—m, 
2g — m — 2, ..., m, m — 1, ..., —m, —m — 2, ..., —2g + m. Let Vg ,..., V-g C F be the 1- 
dimensional subspace on which pm acts by scalars with corresponding weights (in order). 
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Since the quadratic form (•, •) on is preserved by G(0), we have Vj C unless i = —j, 
i.e. unless their weights sum up to zero. Note that pm acts on Nm with weight —2. This 
implies Nm{Vi) = Vi-i ioi m < i < g and —g < i < —m, and that NmiVi^i) = Vi-i for 
—m < i < m. 

We also write V>n := 0„<j<g Vi- From the description of Nm above, one sees that if 
Ad(/i)r E Nm + 0(l)>-i for some h E G{Q){k), then there exists <zV®k such that 
TTi{W^) = and the flag (O C h~^Vg C h“^V>g_i C ... C h~^V >2 C W^) is m-exact 

(see the dehnition of m-exactness in the paragraph before Theorem I, '1.2p . 

In fact, by the definition of Nm, there exists vi E Vi be such that {vi, Nm{vi)) = 1. With 
it can be given by any of the following two choices = h~^V >2 + (/i“^.ui, ±1), where 
h~^V >2 is a subspace of V and thus oiV ®k. 

Conversely, if there exists an m-exact flag 0 C C ... C <Z V ® k, then there is 
a unique right i?(0)(A:)-coset, say B{{k){k) ■ h, such that VF® = for 1 < f < <7 

and In this right coset, one checks from the definition of m-exactness 

that there are exactly two right C/(0)(/c)-cosets, say U{V){k) ■ h, such that Ad{h)T E Nm + 

In other words, there is a 2-2 correspondence between such m-exact flags and right 
[/(0)(fc)-cosets U{0){k) ■ h satisfying Ad{h)T E Nm + 0(l)>-i. Since fm{N) = when 
X E Nm -|-0(l)>-i (recall = {#U{0){k))~^), we conclude from (14.10 that 

^ ^ #JT[2](fe) " XT,m-lik)) . 

The last equality follows from Theorem l3.2l This proves the first statement of the lemma. 

For the stable Shalika germ, by Lemma 14.51 we have T running over G(0)-orbits in its 
stable orbit, classified by H^{k, Jt[ 2]). When T runs over these orbits, above every fe-point 
of Xx^m, ah isomorphism classes of JT[2]-torsor will appear exactly once in Xx^m- Since 
#H^(k, Jx[2]) = #H^{k, Jx[2]) = #Jr[2](/c), the sum of the number of A:-points in all 
isomorphism classes is exactly ^Jx[‘^]ik). This gives 

r\f, fm) = *Xx,m{k) - i^Xx,m-l{k). □ 


Lemma 4.7. (i) #Xx,m{k) = #Sym^{Cx){k) - q#Sym^ ‘^{Cx){k). 

For 0 < m < g, write am{T) = (—1)"^ Tr{Frob : H^{Jx/k,Qi))- Then 

m 

(a) ij^Sywr{Cx){k) = + ... +q + l)am-m'{T). 

m'=0 

L—J 

m' (9-m + 2m'\ 

(in) am[T)= 2 ^ 9 1 , \am- 2 m'{T). 

m'=0 ^ ^ 

Here (i) comes from properties of hyperelliptic curves; above any rational point D — (m — 
l)(oo) E Xx^m{k) C Pic^(C'r), there will be q^ + q'^~^ 1 rational points on Sym^^CT) 

and q'^~^ 1 rational points on Sym™’“^(C'T), where d = 1{D) := H^{Cx, D) — 1 is the 

dimension of the linear system, (ii) and (iii) are direct applications of the Grothendieck- 
Lefschetz fixed point formula and basic properties of Gadic cohomology of curves, their 
symmetric powers and Jacobians. 
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It is now straightforward to verify that Proposition 14.31 follows from Lemma 14.61 and 
Lemma 14.71 

4.2. Nilpotent affine Springer fiber. We now prove Proposition 14.41 Counting points 
on nilpotent affine Springer fiber^, just like counting points on nilpotent Springer fibers, 
is usually purely combinatorial. Let i?°PP c G be the opposite Borel to B with respect 
to S. Our essential idea here is that after applying the formula of Ranga Rao m Thm 

1] , we can do “reduction modulo vr” and arrive at an integral over G{0){k) which is left 
invariant by U{0){k) and right invariant by R°pp( 0)(A:). This gives a combinatorial sum 
over U{0){k)\G{0){k)/B°PP{0){k), which is identified with the Weyl group of G(0). 

To begin our proof, the formula of Ranga Rao in our case can be formulated as follows. 
There exists a maximal T-split torus S G G, whose corresponding apartment contains x 
and whose reduction at x is equal to S'(O) C G(0). Moreover after conjugation we may 
assume that the cocharacter pm' ■ Gm/k 'S'(O) corresponds to pm' ■ Gm /f ^ S and that 
Pm’ also acts on N!^ by weight —2. 

Fix such an S and pm'- Denote by O in this subsection the orbit of Nm'■ Write C 0 
be the subspace on which pm' acts by weight i. Then with suitably normalized measure, 
Ranga Rao’s formula says 

(4.2) J{Nm’Jm)= [ ^{X) [ fmiAdCh)X)dhdX, 

J~s<- 2 iF)no JG{F )^,0 

where ’p{X) is an M-valued function on 0 _ 2 (T) and the measure on the first integral is a 
Haar measure on 0 <_ 2 . The space 0_2 can be interpreted as follows: let V be the standard 
representation of G/f, i-e. R is a {2g + l)-dimensional hermitian space over E. Then pm' 
acts on V with weights {2g — m'), {2g — m' — 2),..., m', {m' — 1 ),..., 1 , 0 , — 1 ,..., —m', {—m' — 

2) ,...,(-25 + m'). 

Denote by I^,..., V-g the 1-dimensional Fi-subspace with these weights, respectively. Let 
Qij-, —g < L J < 5 be the 1-dimensional Fi-subspace of 0 which maps Vj to R. One can then 
check 

0-2 = ^ 0i,i © ^ Qi,j 

j—i=l^\i-\-j\>2m' j—i=2,\i+j\<2m' 

and 

0<-2 ~ 5i,j © Qi,j- 

j—i=2,\i-\-j\>2m' 2>3 

Note that Qij is not defined over F unless i + j = 0, but Qij -|- Q-j,-i is always defined 
over F. Now we fix a “valuation-preserving” identification of Uij : Qij Ga /e so that 
is not in 0 jj n 0 (Fi)x,o but uF^{l) is. Let | • | : Fi —)■ M be the extension of the 
standard norm on F, i.e. One then computes 

‘fix) = n \^ijix)\^ X G 0 <_ 2 (t). 

i-*= 2 , J=m'( 2 ), \i+j\<2m' 

In (14.2p . if X G 0<_2 is such that |'Ujj(X)| >g^/^,thenX ^ 0 (T)x,_i /2 and/m(Ad(F)(X)) 
0. Moreover, the value of /m(Ad(F)(X)) depends only on Uij{X) modulo Oe since fm is 
locally constant by 0 (F")x,o- Let’s now denote by Zm’ the image of 0 (F’)x,_i/ 2 n 0 <_ 2 (F’)nO 

just make up this term to refer to nilpotent p-adic orbital integral. However nilpotent Springer fibers 
do mean Springer fibers over nilpotent elements. 
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in g(l)(/i;) and by the push-forward of the measure to the 

finite set Also let dh be the push-forward of the measure dh from G(F)x,o to G{0){k). 
Then we can rewrite (I4.2p 

(4.3) [ [ U{Ad{h)X)dh-{d^)^,, 

JXGZ^, JG{0){k) 

We can similarly define Vg,...,V-g as 1-dimensional A:-subspace on which pm' acts by 
strictly decreasing weight. In fact Vi is just the line spanned by Vi in the previous subsection 
(with m replaced by m'). We can then define uij : Q{k) ^ k in the same way, scaled so 
that if we write the reduction maps redi : q{F)^ _i /2 0 ( 1 ) and red 2 : 7r~^/^OE k, then 

rtij(redi(A)) = red 2 {uij{X)) for X G 0(T)x _i/ 2 - 

Lemma 4.8. If fm{Ad{h){X)) ^ 0 for X € Z'^, then Uij{X) ^ 0 when j — i = 2 and 
\i + j\ < 2m'. 

Proof. The function fm{X), as a function of A G 0 (l)(/c), is invariant under conjuga¬ 
tion by U{0){k). The assertion of the lemma, for X G Z^, is a property that is pre¬ 
served under conjugation by i?°PP(0)(/c). Therefore it suffices to consider /i in a set of 
representative for U{0){k)\G{0){k)/B°'^^ (0){k), which can be taken to be the Weyl group 
NGio)ik){Smk))/S{0){k). 

Identify Sg k {±1}^ with Ac'(o)(fc)(5(0)(fe))/5(0)(fe) in the following way: the first com¬ 
ponent Sg shall permute V^,...,I/i) and the i-th {± 1 } in the second component switchs 
Vi and V-i. We now check directly the assertions for all a & Sg « {±1}®. To have 
/m(Ad(cr“^)A) / 0 for some X G Z^', it’s necessary that Ad{a)Nm G Z^'. This hap¬ 
pens exactly when 

Condition 4.9. Consider u G S'^ k {±1}® acting on {0, ±1,..., ± 5 } where Sg permutes 
{ 1 , ■■■,§} and { — 1 ,..., —g} simultaneously, the i-th component in {± 1 }® switches ±i, and 0 
is always fixed. Now for any —g < i < j < g, 

(i) For j — i = 2, \i + j\ < 2m, either a(j) — a{i) = 1 and |cj(i) -|- cr{j)\ > 2m', or 

O'O') - o'(i) > 2 . 

(a) For j — i = 1, 2m < -|- jj < 25 - 1 - 1 , the same condition is required. 

It’s straightforward to see that the condition is satisfied only when a G Sg, i.e. a 
preserves {1, ...,g}. One then see inductively that cr”^(0) = 0 => cj“^( 1) = 1 => cr“^( 2 ) = 
2 ^ ..., until a~^{m') = m'. We conclude that a and thus a~^ preserves Vi, ...,Vm>. Since 
tti_i^i+i(Am) / 0 for i = 0 , ...,m' — 1 , this implies that rij_i^j+i(A) / 0 for the same i’s, 
which is what we have to prove. □ 

On the subset C Z^' where the conclusion of the lemma holds, {dp)m' is nothing 
but a multiple of the counting measure. We’ll pretend it is exactly the counting measure 
and discuss the normalization constant later. The idea in the lemma can now be further 
applied to compute the integral; write 

E f^{Ad{h)X). 
x^z°, 

m' 


We have to compute Im'W (up to normalizing constant). Exactly as in the 

heG{0)ik) 

situation of the previous lemma. This function is invariant under left translation by 
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U{0){k) and right translation by i?°PP(0)(A;), and we arrive at a sum over the Weyl group 
Sg K {±1}3. To have / 0, a needs to satisfy Condition 14.91 

Denote by C Sg the set of such a. For a G Ern,m'i the number of elements 

in the double coset 17(0)(A:)cr“^i3°PP(0)(/c) is given by #B{0){k) ■ Also the sum 

where 

5i(o-) = 5^ - #{1 <i<j<9\ <T(i) > cr(j)}. 

S 2 {a) = m-m' + #{1 <i<j <g\ a{i) > a{j), j -i> 1}. 


We also have to figure out the normalization of measures. The single choice X = Nm' G 
Z^, C 0(1)(A;) and/i = id ^ G{0){k) correspond to the lattice 0(F)x,i/2/(0(T)x,i/2n0^ J C 
q{F)/qj^ (F). a careful inspection of the normalization at the end of Appendix Rl shows 
that this lattice is to have measure 1. 


In addition, as Ranga Rao’s method begins with Iwasawa decomposition G = i3-G(F’)x,0) 
we also have to divide by how much they intersect, namely the order of (G(T)x,o C 
R)G(F)x,i/ 2 /G(T)x,i/ 2 ) which is #R(0)(/c). In summary, all the way from (I4.3p we have 

(4.4) J{Nm'Jm)= Y. 

qm—m'—#{l<i<g \ (7{i)>a{i+l)} 

Proposition 14.41 now follows from Proposition IB. 71 This completes the proof of Theorem 

KT\ 



4.3. More Shalika germs. Recall we had (in the beginning of Section[3|) jm ■ Sym'”(C' 7 ’) —>■ 
Pic^(C'r) and x2 ; Tp —)• Pic^(C'r). The latter map is etale Galois with Galois group Jt[ 2]. 
Denote by Sym (Gt) ■= Sym™(C'r) Xpici(CT) fiber product, which is an etale 

J'r[2]-cover of Sym™'(C'r). We have 


Theorem 4.10. For 0 < m < g, we have 


^nJT) = 


1 


Y #Sym {CT)ik) ■ ■ Gti-g+ m - 2£ + 1) 


. 0<2£<m 


#JT[2](fc) 

(^ + 1 ) Y 




\GT){k)-q^ -Gi^-gFm-U) \ . 

0<2£+l<m 

See Definition \B.1\ for the combinatorial numbers Gi{-). When m > 0, for the other 
nilpotent orbit in the stable orbit of N^, simply change Ct to its quadratic twist. 

Proof. Write in this proof 


u = 




/JifjoX 



, v = 

JiT,h) 

, w = 

^n^T) 

\ y 

\ ... y 


#Sym {CT){k) 
#^\CT){k) 


V 


/ 


We have to write u in terms of w. The first half of Lemma 14.61 says 
Mfm) = ^jj^2]{k) " #XT,m-l{k)) 
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and a JT[2]-cover version of Lemma l4.7f ii gives: 

#XT,m{k) = ^SyTxT {CT){k) - q ■ Sym™ ^(C't)(A:). 


Putting together, they imply v = where and {Blj^)o<i,j<g are 

lower triangular matrices with 




if i = j. 

if i = j + 1. , B^f 


otherwise. 


1 if i = j. 

-q iii= j + 2. , 
0 otherwise. 


To recover r^^(T) from Jf{fm), i-e. compute u in terms of v, we need to “invert” 
Proposition 14.41 One observe that Proposition 14.41 is the same as saying v = B^^^B^"^^Au, 
where A is the matrix in Proposition IB.61 with x = g, and 




g* ^ if i > j 
0 otherwise. 


Proposition IB.61 says 

{A-% = 


q^Ci{-g + j) if i = j + 2£, £ G Z>o 
0 otherwise. 


And we have u = A ^B^^'>B^"^^w. One observe that B^^\ and B^^'> 

all commutes, and := is given by 


{B^%3 


1 if i = j 

-(g + 1) ifi = j + l 
q \i i = j + 2 

0 otherwise. 


We have u = A By the formula for A ^ and B^‘^\ we see that if i = j + 2i, 

i E Z>o, then {A-^B‘^'^^)ij = q^Ci{-g + j) + q^Ci_i{-g + j + 2) = q^Ci{-g + j + 1) by 
Proposition IB. 31 with C'_i(-) understood to be zero. And A i = j + 2£ + 1, £ G Z>o, then 
{A~^B^'^'>)ij = —{q + l)q^Ci{—g+j + l). This is essentially the content of Theorem ITTOl □ 


Another way of thinking of these covers Sym (Ct) is as follows. Denote by ot the 
JT[2]-torsor (x2)~^(oo). Then Sym {Ct) is the etale J^pj-cover of Sym™'(C''r) for which 
the fiber above m(oo) is isomorphic to ax- There is a unique such one since Jt[2] is the 
maximal abelian 2-annihilated quotient of 7r^*’*“”^®(Sym”^(C''r)). 

In the rest of this section we suppose n = 2g + 2. So that G = U 2 g+ 2 {E/F) is instead 
an even quasi-split unitary group (still ramified). Recall we can take T E 0(T)x,_i/2 to be 
any lift of T E s(T)x,_i/2/0(T)x,O) which is regular semisimple and is associated the genus 
g (projective smooth) hyperelliptic curve Ct = (y^ = {—1)^^^pt{x)) where pt{x) is the 
degree 2g -|- 2 monic characteristic polynomial of T. 

We use then Theorem 13.101 instead of Theorem 13.21 to obtain the following result. Let 
Ai, ...,\g,X'g again be the Frobenius eigenvalues on H^{CT/k,Q£) that AjA' = q. We 
also put artificially that Aq = 1, Aq = q. Write this time (note the difference on the range 
of S with the odd case) 
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MU :=(-!)”■ E (nA + ^O). 

Sc{0,...,g},\S\=m Vies / 

For 0 < m < g + 1, let be any element in any nilpotent orbit in g = Lie G with two 
Jordan blocks of sizes 2g + 2 — m and m (a regular nilpotent if m = 0). There can be either 
1, 2 or 4 of such orbits. 

What we can show in parallel to Theorem 14.11 using the method in this section, is 

Theorem 4.11. The stable Shalika germs at T for nilpotent orbits with two Jordan blocks 
of an even quasi-split ramified unitary groups is 

T% (T) = a^iT). 

-'''m 


As for general (non-stable) Shalika germs, we encountered a technical difficulty: what 
was developed in Theorem 13.101 only allows us to compute Shalika germs for nilpotent 
G'“‘^(F)-orbits, where (5“*^ = PU 2 g+ 2 {E/F). The image of G{F) —)• G°‘'^{F) has index 2 in 
G°‘'^{F). If we take u G G°‘^{F) to be any element outside the image, then what we can 
compute is the sum of Shalika germs T^^(T) -h r,^d(«)iv^(^) = + T 

To state what we are able to obtain in parallel with Theorem 14.101 with the geometry 
from Theorem 13.101 we need a notion about nilpotent orbits of G = U 2 g+ 2 {E/F) with two 
even Jordan blocks. 

Definition 4.12. Let {{2g -|- 2 — m,m),di,d 2 ) for 0 < m < g 1 with m even (resp. 
{{g + 1, g + l),d) for m = g 1 if g -\- 1 is even) be a nilpotent orbit with two even Jordan 
blocks. We say the orbit is hyperbolic if did 2 = —1 (resp. d=l), and elliptic otherwise. 

We also say any nilpotent orbit with two odd Jordan blocks is hyperbolic. They are 
characterized by the following: for any two distinct nilpotent orbits, both having two Jordan 
blocks, one lies in the closure of the other if and only if they have different dimensions and 
they are either both hyperbolic or both elliptic. 


As in Theorem 14.101 we also need notations about covers of Gt. Recall we have two 
rational points G Gxik) (see subsection 13.2p . Fix a choice of any of them, say 

oo^^V Consider (x2)“^(oo(^^), where x2 is the etale JT[2]-Galois map in Theorem l3.9l This 
is a JT[2]-torsor, which we shall denote by ot- 

Consider also G Jrik). We have, by Lang’s theorem, JT{k)/2JT(k) = 

H^{k, Jr[2]). Denote by /3r the Jr[2]-torsor that are given by in this wajQ. For 

even non-negative integers m, we write Sym (Gt) the etale Jt[ 2]-cover of Sym™'(C''r) for 
which the fiber above ^(oo^^^)-!- y(oo(^^) is (as a Jr[2]-torsor) isomorphic to ar- Write also 
Sym ’ (Gt) the etale JrpJ-cover of Sym”^(C''r) for which the fiber above y (oo*^^))-|-y(oo*^^)) 
is isomorphic to ax 

For odd m instead, we write Sym {Gt) the etale JT[2]-cover of Sym™'(CT) for which the 
fiber above (oo^^)) is isomorphic to dT- And we write Sym ’ (C^) the etale 

JT[2]-cover of Sym'"(CT) for which the fiber above -|- is isomorphic 

to ar x'^rPl I3t. For all m, we write S'^{Gt) = (jfSym {GT){k) + #Sym ’ {GT){k)'j. 

^In fact, (oo*-^^) — (oo^^^) £ 2JT{k) except when all irreducible factors of Pt{x) G k[x\ are even and 
n = 2g -f 2 is divisible by 4. Consequently, if there is an odd factor of Pt{x) or if g is even, Jt is trivial. 
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Lastly, we write C'rp to be the quadratic twists of Ct, so it has two points above infinity 
that are not defined over k. Note Pic°(C'^)[2] = Pic°(C'T)[2] = Jt[2]. For 
m even, write Sym {C'rp) the etale JT[2]-cover of Syni™'(C'y) for which the hber above 
is isomorphic to ot- We also write Sym ’ {C'p) the the etale JT[2]-cover 
of Sym™'(C'y) for which the fiber above + ^{ 00 ^“^^) is isomorphic to ot I3t- 

And we write 5™’(C'^) = f#Sym {C'p){k) + #Sym ’ {C'p){k)\. 

Note = 2^ where r is the number of irreducible factors of pt{x) in 

k[x]. We have 

Theorem 4.13. For 0 < m < g + 1, let be any nilpotent orbit with two Jordan blocks 
of sizes 2g + 2 — m and m. Recall u E G°‘^{F) is any element that doesn’t come from G{F). 
We have 

If Nm is hyperbolic, then 

( s'“-“(CT)-9'-fc,(-9 + m.-2(> + l)-(^+-^fC,_i(-9 + m-2f+l)') 

\0<2£<m y/Q / 

-2{q +1) ^ • Ce{-g + m - 2^) | . 

0<2£+l<m / 


If otherwise Nm is elliptic, then 

''jv„(r) + r;i„(<.<iWT) = 2-(’-). 

+ m — 2i + 1) + {,/q H—+ m — 2^ + 1) 

0<2e<m ^ 

Here we adapt the convention that Sym = Sym = 0. For case m = 0 the two 
formulas agree. 



5. Endoscopic transfer of nilpotent orbits for ramified unitary groups 

In the beginning of this section and subsection 15.21 i.e. except in subsection 15.11 we’ll 
assume char(E) = 0 so that the endoscopic transfer of Langlands-Shelstad [9] is valid. 
We however note that we can actually also work with sufficiently large char(E) (in an 
un-effective manner) thanks to Gordan-Hales [5]. 

In [T], Assem stated a conjecture regarding stable distributions supported on the nilpotent 
cone for a reductive p-adic group. Recall that a distribution T> E C^{q{F))* is called 
stable if T>{f) = 0 for every / E Cf°{Q{F)) with the property that J^*(A,/) = 0 for all 
X E r"(F). 

For quasi-split unitary groups, Assem’s conjecture asserts that all stable distributions 
supported on the nilpotent cone can be written into a linear combination of stable distri¬ 
butions where each term is a linear combination of nilpotent orbital integrals on various 
orbits in a single stable orbit. Moreover, on each stable nilpotent orbit there is a unique 
(up to constant) linear combination of the orbits for which the integral becomes stable. 
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Assem also had a conjecture regarding endoscopic transfer of nilpotent orbits. For en¬ 
doscopic transfer of unitary groups, relevant elliptic endoscopy groups are products of 
two quasi-split unitary groups x Un 2 {E/F), while the target of endoscopy is 

Uni+n 2 {E/F). Here the three unitary groups split over the same quadratic extension. 

Recall that if H (e.g. H = Uni{E/F) x Un 2 {E/E)) is an endoscopy group for G with ^ = 
Lie H, then the transfer conjecture (for the Lie algebra) asserts that for any / G C'^{q{F)), 
there exists a function G C^(^{E)) such that 

K{Y)J{Y,f) = r\XJ^), VA G 


Here X is G-regular |21t 2.2] and Y runs over regular semisimple orbits in 0 (F) that 
“matches” with Y. Also k = kh is some character (determined by H) on the set of orbits 
of such Y. 

The transfer conjecture was proved by Waldspurger [20] conditional on Ngo’s later mar¬ 
velous proof m on the fundamental lemma. Given the transfer conjecture, for any stable 
distribution V on C'^(1)(F)), we can define its endoscopic transfer to be the distribution 
T>^ : f I—)• T>{f^). It’s obvious that such distributions has to be G'(F)-conjugation invariant. 

If F is a stable distribution supported on the nilpotent cone of f), i.e. it’s a linear 
combination of nilpotent orbital integral that becomes stable, then has to be also 
supported on the nilpotent cone. It thus makes sense to talk about endoscopic transfer of 
nilpotent orbital integrals. 

In |2T], assuming p large enough, Waldspurger completed the study of stability and 
endoscopic transfer (classical endoscopy, in the sense of Langlands-Shelstad |9]) for nilpotent 
orbital integrals for unramified classical groups. In particular, Assem’s conjectures (see e.g. 
Conjecture 15.11 and 15.5|) were proved in these cases. 

The endoscopy data and the transfer factor, etc, are computed in m Chap. X]. These 
data as well as Waldspurger’s formula can be equally stated when F/F is ramified. The 
main goal of this section is to show that Theorem 14.1114.10114.111 and 14.131 provide special 
cases and evidence on that Waldspurger’s result could equally holds for ramified unitary 
groups, as well evidence for Assem’s conjecture. 

5.1. Stability. We state Assem’s stability conjecture in the unitary case. 

Conjecture 5.1. [U Conj. C, pp. 2] Let F be a non-archimedian local field with char{F) = 
0 or char{F) ^ 0. For every stable nilpotent orbit O of a quasi-split unitary group 
Un{E/F), there should be (up to constant) a unique linear combination of orbital integrals 
among the orbits in O that gives a stable distribution. All stable distributions supported on 
the nilpotent cone can be written as a linear combination of such stable distributions. 


Waldspurger gave explicit formula for these combinations. If we restrict our attention to 
nilpotent orbits with (at most) two Jordan blocks, the formula of Waldspurger is simplified. 
For 0 < m < ^, denote by Om(0) the set of nilpotent orbits with two Jordan blocks of sizes 
n — m and m (or regular nilpotent if m = 0 ). 

Theorem 5.2. (Waldspurger, [211 IX.15]J Suppose E/F is unramified, char{k) > 3n-|- 1, 
and char{F) = 0. Then 
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(a) For any 0 < m < ^ with 2\mn, 

J{Nm,-) is a stable distribution. 

NmeOmiO) 


(b) For any 0 < m < | with 2 f mn, we have = 2. Denote by Nm'^ and 

these two orbits, then 

is a stable distribution. 


( 2 ) 

m 


What we can prove using Theorem 14.11 and Theorem 14.111 is 

Theorem 5.3. Suppose instead E/F is ramified. Under the assumption char{k) 2 and 
either char{F) = 0 or char{F) > n, we have 

(i) The same results in Theorem \5.^ are true for m <2. 

(a) The same results in Theorem \5.<i\ are true for all m assuming Conjecture 15.11 of 
Assem. 


Proof. We take I an even integer so that N is conjugate to tt^N for every nilpotent N E q{F) 
(see e.g. m Sec. 3.1]). Most of the time (e.g. when char(T) = 0) i = 2 works. 

Let T E 0 (T)x,_i /2 be any lift of T E 0 (l)^®(/c) as in the introduction. The theorem of 
Shalika states that, for every function f E C?°(n(F)), there exists AL such that > Nn, 
if we write fiN){X) = /(vr^^X), then 

^ rg(r)j(o,/(^)). 

OeO{0) 

Nilpotent orbital integrals have the property (due to the symplectic structure on O) that 
J( 0 ,/( 7 V)) = Q 2 J{0,f). This allows us to rewrite 

(dimG-rk^G)/2 

/) = r\f, /(^)) = /). 

d=0 OGC)(0),dimG=2d 

Now let / be any “stable” function; J^^{X,f) = 0 for every X E 0 ”®(F). The LHS by 
very definition vanishes. Interpolating with enongh different N, we see that for every d, 

^ rg(T)j(o,/) = o. 

OGO(0),dimG=2d 

In other words 

Tq{T)J{0, •) is a stable distribution. 

OGG(0),dim 0=2d 


When d = (dimG — rk^G)/2 — m with m < 2, the only nilpotent orbits with dimension 
2d are those nilpotent orbits with two Jordan blocks of sizes n — m and m (or one with size 
n if m = 0). To use previous results on Shalika germs, we need 

Lemma 5.4. For any 0 < m < there exists T E 0 (l)”'^(/c) such that am{T) 0. 
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Proof. We use a parity trick. For each 0 < m < §, we claim the existence of some T for 
which OmiT) is odd. When m = g + 1 and n = 2g 2, we observe that has the 

same parity as thus we reduce to the case m = g. 

The idea is that the hyperelliptic involution gives an involution on Sym”*(C'T)(A:). The 
parity of #Sym™'(C''r)(fe) is thus given by the number of fixed points that are defined over k, 
which in terms depends on the Galois structure on the Weierstrass points, or equivalently, 
how the characteristic polynomial factors in k[x]. 

Using Lemma 14.71 (ii),(iii) one can show the following: take T so that pt{x) is an irre¬ 
ducible separable monic degree n polynomial. Take T' so that pt' {x) is another separable 
monic polynomial with two irreducible factors of degree m and n—m. Then am{T) ^ am{T') 
(mod 2 ). □ 


Theorem 14.II and Theorem 14.Ill give us r^(T) = OmiT), or —am{T) for one of the orbits 
if both m and n are odd. This completes part (i) of the theorem. For part (ii), simply note 
that Conjecture 15.11 allows us to separate nilpotent orbits with two Jordan blocks out (or 
nilpotent orbits of any type of Jordan blocks) for stability question. □ 


5.2. Endoscopic transfer. The flow of this subsection is parallel to the previous section. 
However we will encounter interesting geometric and combinatorial identities that can be 
thought as consequences of endoscopy. Recall that our endoscopy group of G = Un{E/F) 
is UniiE/F) X Un 2 {E/F) with ni + n 2 = n. We write gi = Lie Uni{E/F) and 02 = 
Lie Un^iE/F). 

We begin by stating the corresponding conjecture of Assem. The original conjecture 
of Assem for endoscopic transfer of nilpotent orbits comes from an induction construction 
due to Lusztig and makes use of the Springer correspondence (see [H 4.3]). In our case 
Un^{E/F) X Un 2 {E/F) is isomorphic to a twisted Levi subgroup of Un{E/F), and the 
construction agrees with that of Lusztig and Spaltenstein m- 

We summary their construction: Let G be a reductive group over an algebraically closed 
held F and M a Levi subgroup. Take P = MN C G any parabolic subgroup for the 
Levi, where N is its unipotent radical. For any nilpotent orbit O of Lie M, the variety 
O ■ Lie N has a dense open subset contained in some nilpotent orbit O' of G. We then 
denote ind^O := O'. In general when the reductive groups are dehned over F, this should 
be understood as an induction between stable orbits. 

In our case, G = Un{E/F) and M = Urn{E/F) x Un 2 {E/F) with ni + n 2 = n. The 
induction for nilpotent orbits with two Jordan blocks is especially clear: if and 
are stable nilpotent orbits in 0 i (resp. 92 ) with two Jordan blocks of sizes (ni — mi, mi) 
and (n 2 -m 2 ,m 2 ) where 2 mj < m, then x = ^m, the stable 

nilpotent orbits with two Jordan blocks of sizes (n — m, m), with m = mi + m 2 . 

We now assume Conjecture 15.II Consider any stable combination Pi of nilpotent orbital 
integrals of Uni{E/F) on orbits in a stable nilpotent orbit Oi C 0 i(T), and likewise another 
stable combination V 2 of Un 2 {E/F) on orbits in a stable nilpotent orbit O 2 C 02 (T)- They 
give a stable nilpotent distribution Vi (g) V 2 on 0i(T) x 02 (T) by Vi (g) P 2 ifi (g) / 2 ) = 
'Dl{fl)P2{f2). 
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Conjecture 5.5. [U Conj. D, pp. 83] The endoscopic transfer of'Di®'D 2 given above 
is a linear combination of nilpotent orbital integrals on orbits which lie in the stable orbit 


ind 


Un{E/F) 

U„^{E/F)xUr,2iE/F) 


(Oi XO2). 


As in Theorem 15.21 Waldspurger proved the conjecture in the case of unramified classical 
groups. We formulate some cases of Waldspurger’s result with two Jordan blocks. Let 
0^1(0) be the set of nilpotent orbits of Un^{E/F) with two Jordan blocks of sizes ni — mi 
and mi. Similarly and Om(0) are used for nilpotent orbits of Un 2 {E/F) and 

Un{E/F). We have 

Theorem 5.6. (Waldspurger, |21l XII.9] j Suppose E/F is unramified and char{k) > 3n+l. 
Then 

(a) Suppose ni is odd and n 2 is even, so that n = ni + n 2 is odd. Fix 0 < 2mi < ni, 
0 < 2m2 < n 2 and write m = mi + m2. Write e G F^/N^/pE^ for the non-trivial class. 
For nilpotent orbit Nrn £ C’m(O), put 'y{Nm) = —1 if'm is odd and Nm is the orbit classified 
by ((n — m, m), (—1)^). In all other cases put 'y{Nm) = 1- We define likewise the 

factor 7(iVmi) for Nmi G 0^1(0) • Then 

XmeOm(o) 

is the endoscopic transfer of the stable distribution 

^ ^ 'y ^ J {Hmn') ® J {Em2 T') ■ 


(b) Suppose both ni and n 2 are even, so that n = ni+n 2 is also even. Fix 0 < 
0 < 2m2 < n2 and write m = mi + m2. For any Nm G 0^(0), put '^{Nm) = 
hyperbolic (see Definition\4.12( and 'y{Nm) = —1 if Nm is elliptic. Then 


2mi < ni, 

1 if Nm is 


Y l{Nmr^J{Nm,-) 

NmeOmiO) 


is the endoscopic transfer of the stable distribution 


J{Nm^,-)®J{Nm2E)- 


Parallel to Theorem 15.31 what we can show using Theorem 14.1114.10114.111 and 14.131 is 


Theorem 5.7. Suppose instead E/F is ramified. Then 

(i) The same results in Theorem \5.6\ are true for m = mi + m2 < 2. 

(a) The same results in Theorem IM are true for all mi, m 2 assuming Conjecture I5.il 
and Conjecture 15.51 

Proof. We only give the proof for case (a). The proof for case (b) is completely the same 
while replacing the role of Theorem 14.11 and 14.101 by Theorem 14.111 and 14.131 The reason 
that in case (b) we want to assume both ni and n2 are even (instead of only n = ni + n 2 
is even) is that in Theorem 14.131 we are only able to compute Tjv^CT) + T f^^{ad{u)T). It 
happens that this discrepancy matters exactly when ni is odd. 

The idea is similar to the proof of Theorem 15.31 Let xi be a vertex on the Bruhat-Tits 
building of Uni{E/F) with reductive quotient SOm/k- Let Vi be the quasi-split quadratic 
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space which is the standard representation of this SO^i- Let Ti be any regular semisimple 
self-adjoint endomorphism of Vi. We have the same notations for Un2{E/F) and let T2 be 
any regular semisimple self-adjoint endomorphism of V 2 - 

Let pti{x),pt2{x) E k[x] denote respectively the monic characteristic polynomials of Ti 
and T 2 . We assume that pti{x) and pt 2 {x) are coprime. Write Ct^ = = pti{x)), 

Ct 2 = (y^ = PT 2 {x)), Jti = Pic*^(C'ri) '^'^2 = Pic°(C'r 2 )- Py abuse of notation (as 
we don’t have T yet), we also write pt{x) = PTx{x)pt2{x) a degree n monic polynomial, 
Ct = (y^ = Pt{x)) and Jt = Pic°(C'T). 

The G(0)(fe)-orbit of actual such T E 0 (l)^®(/c) with characteristic polynomial pt{x) is a 
torsor under H^{k, Tr[2]). This torsor is in fact canonically trivial [21 Prop. 4] as mentioned 
in the introduction; the identity element in H^{k, corresponds to the T for which 

(x2)“^(oo) C Ft is a trivial JT[2]-torsor iTheorem 13.11 [21 Prop. 4] and [221 Cor. 2.5 and 
Prop. 2.29]). This orbit of T is also the one that intersects with the Kostant section [3l 
Sec. 7]. From now on we’ll use the symbol T to denote a representative of this orbit for 
which (x 2 )“^(oo) is trivial. 

Let Ti E 0 i(F)xi,_i /2 be a lift of Ti and likewise for T2. The orbits of those T E 0 (F) that 
“matches” with (Ti,T2) E 0 i(F) x 02 (F), i.e. that has the same characteristic polynomial, 
enjoy a one-one correspondence with those orbits of T classified by H^{k, Jr [2]) in the last 
paragraph, thanks to Lemma 14.51 

Recall that Jt[2] = ^ 2 /P 2 — ker(Res^^^^^^’^^^V 2 ^ 2 )- In the middle 

group the p2 is embedded into Res^^^^^^^^^^y 2 via the diagonal embedding. The second 
group and the third group are also dual to each other; this gives a self-dual structure 
Jt[2] X Jt[2] —)• P2- 

l\T f \ f \ U klx]/pT(x) T5 klx]/pTRx) klxl/pT^ix) 

Now as pt[x) = pTi{x)pT2[xj, we li-me Res 712 = Res^ /X2xRes^ 72- 

On the latter group that is an element k = Kni,n 2 •= (1) “!)• Since degpT 2 = ^2 is even, 

this element lies in F°(fc, ker(Res^^'^^^^^^^V 2 —^ P2)) — F°(fc, Jt[2]) = H^{k, Jt[ 2 ]*)* = 
H^{k, Jr [2])*. In other words, k defines a character on H^{k, Jr [2]). 

By carefully checking the transfer factor, one can conclude that 

^ K{a)J{fa,-) 

aGHRfc.Jrp]) 

is the endoscopy transfer of 

J^*(fi,.)® J^*(r 2 ,-)= j(r,,,-)® J(Ta 2 ,-). 

[ 2 ]) OL 2 ^H^{k^JT 2 [ 2 ]) 


Here Tq, E 0 (F) is any representative of the orbit classified by a as described, and 
similarly for Tq-j E 0 i(F), Tqj E 02 (F). Arguing as in the proof of Theorem [FS] and assume 
Conjecture 15.11 and 15.51 if m > 2, we have 


K{a)r^jf^)\j{Nmr) 

NmGOmiO) \aeHRk,JT[2]) J 


is the endoscopy transfer of 
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E E 


E 


■ps£ 

iV„ 


(Ti)r^^_ (T 2 ) • ® •)■ 


mi + m2 = m iV„ 
0 < 2mi < ni 
0 < 21712 < ^2 


^eoi (o)7Vi. 


• 260 ^ 2 ( 0 ) 


Later we will simply write mi + m 2 = m for the first summation in the last formula 
while it should be understood that mi and m 2 vary only in the range for which and 
Nm 2 are defined. The key is to prove 


Proposition 5 . 8 . We have equality 

a&W{k,JT[2]) mi+m2=m 

where in the summation in the RHS, Nmi is chosen arbitrarily in O^^(O) and Nm 2 is 
chosen arbitrarily in O'^^{0). See the definition of'y{-) in the statement of Theorem \5.6[ 


Proof. Using Theorem 14.11 ITTOl and H.lll what we have to prove is the following geometric 
identity that nnderlies this endoscopic transfer: 


(El) 

a^2 (E2) = 

mi+m2=m 


1 


#Jt[2](/c) 


(5.1) ( K{a) 

yQ^m{k,jT[2\) 

-(g + l) Y 

0<2£+l<m 


Y ^Sym”" ^\cTj{k) ■ {-qY ■ Cfi-g + m - 2£ - 1) 

:2l<m 

' —^ m—2£—1 

#Sym {CT^){h) ■ {-qY ■ Ci{-g+ m-2i) 


We have to explain the slight abuse of notation here. Different a G Lf^(A:, Jr[2]) gives 
us the same Ct^ = Ct- However, the definition of the etale JT[2]-cover Sym {Cto) 
of Sym™’(Cr) depends on the orbit of Tq, thus depends on a. In fact, changing a G 
H^(fe,Tr[2]) exactly amounts to changing the Probenius structure on Sym {CtY) as a 
JT[2]-torsor over Sym™'(C'r). 

Recall that T is used to denote the Ta with a trivial. For any k,' G H^{k, Jt[2])* = 
H^{k, Jt[2,]*), we can consider the K'dgotypic component H*{Sym (Ct^))^/. We have 

Tr(Frob : H*{^'"{CTj/-k,Qi).') = «^'(«) ' Tr(Frob : H*{Ct)/^^ 


Summing over all a and all a', we see 

Y i^{a)ifSyur {CT^){k) 

a&m{k,.JT[2\) 

(5.2) = (-1)”^ Y '«(«) Y «'(«)• Tr(Frob : H*(Sym™(CT)/fc,Q£)K')- 

a&m{k,JT[2\) K'&m{k,JT[2\)* 

= {-ir#JT[2]{k) • Tr(Frob : H*{^" {Ct))J. 


In the last step we used the equality ffH^{k, Jr[2]) = #Jr[2](/c). To compute Tr(Frob : 
JI*(Sym (Cr)/fc, Q^)^), it will be a good idea to first deal with the case m = 1. In the rest 
of the proof we write Ct ■= Sym {Ct). This is an etale Jr[2]-cover of Ct- 
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Finite covers between (projective smooth) curves can be read out from their function 
fields. Let’s base change from the ground field k to k for the moment. Recall Ct is a 
double cover of Their function fields are respectively k{x) C k{x, ypri^)- The key is 

to observe 

k{CT) = k{x, \/pt{x), \JP{x) I P{x) runs over even degree divisors of pt{x)). 

This is because the above function field extension gives an etale Jr[2]-cover of Ct, which 
is unique over k. Now k, being a non-trivial element in J'r[2](A:)*, corresponds to a degree 
2 cover of Ct inside Ct —>■ Ct- This cover is given by the function field k{C^) = 

Hx, \/pt{x), VmM) = Hx, ^ypTl{x), Vpt2{x))- 

The curve as well as its function field, descend back to k. Precisely, since Ct is defined 
to be the curve for which the fiber above oo is trivial, we have C^ = k{x, \/pt{x), Vkt^T^) 
(here it’s important that pt 2 {x) was chosen to be monic). Now recall 

Lemma 5.9. Let X be a quasi-projective variety over any field k and G be a finite group 
acting on X. Choose prime (. which is coprime to |G|. Let Y = X/G be the scheme-theoretic 
quotient. Then = H*{X/. 


Using the lemma, we have 

H*iCT/k,Qe)n = H*iC^/-,,Q,)QH*iCT/-k,Qi), 

where the two sides of the equality are in the abelian category of virtual representations of 
the free abelian group generated by Frobenius. Nevertheless, it’s obvious from the function 
field of Cy that it is a (/X 2 )^-cover of P^, and that the three double covers in the middle are 
Ct, Cti and Ct 2 ^- This gives 

H*{C:^/-„Qe)QH*{CT/-k,Qe) 

= {H*{CTj-k,Qe) e ® {H*iCT2/-k,Qi)e H*{F^/-„Qi)) 

= H\CTJ-„Qe)®H\CT2/-k,Qi)■ 

ln summary H*{Ct/ k,Q.i)K, = H^{CTi/k,Q.d ® H^{CT 2 /k,Q.d- For general m, what we 
have is 

Lemma 5.10. Let Jti and be the Jacobian of Cti and Ct 2 , respectively. Then 

m 

(5.3) H*{S^""iCT)/k,Qe). = ^H^iJTjk,Qi) ® Qr)- 

d=0 


To prove Lemma [5.101 note that 7r)^*’*“™'®(Sym™’(Cr)) = 7r^*’*“”^'^(C'r) canonically, and 
thus we have a double cover Sym"*(Cr)'’’ —)• Sym™’(Cr) corresponding to Of — >■ Ct- This 
double cover can be seen as a 5m-quotient of {{Ct)^Y (C't)™'; the “diagonal” double 

cover in the (/i 2 )™'-cover (C^)™ —)• (Ct)"^- 

Now the cover {{Ct)'^Y is a ((^ 2 )”^ x ^ 2 )-cover of (P^)™'. For any u G {1,2}™', denote 
by the “diagonal” double cover of (P^)™ in (P^)™. These are exactly 

all the double covers of (P^)™ which are between {{Ct)^)'^ (P^)™ but not between 

(Ct)"^ — )• (P^)™. We thus have 

H*{{{CTrr/k,Qe)eH*{{CTr/kM = Y. {H*{Vy-„Qe)e H*{{F^r/-„Qe)) - 
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On the other hand, for the (/r 2 )™'-cover we can consider the product 

map (p : /r™ —)• /U 2 - Then the (/>-isotypic part is equal to the term in the previous sum: 

m 

= Vi.G{i,2r. 

i=l 

Nevertheless, from the Kiinneth formula one deduces -f^^(C*^^,(i)/fc,Q£)■ 

Putting together, we have 

m 

H*{{{CTrrh,Qi)eH*{{CTr/k,Qe)= E 

ve{l, 2 }"> i=l 


Now the LHS of (j5.3p is the S'^-invariant part of the LHS above, taking ^m-invariant on 
the RHS gives 

m 

d=0 


where on the RHS, d correspond to the number of i with v{i) = 1. Here (!) the Sym'^H^ 
above has the meaning of the d-th symmetric power of (virtual) representations as super 
(i.e. (Z/2Z)-graded) vector spaces; that is, Sym'^dd^ = in the usual notation. This 

proves (ISSI). 

Combining (15.21) and (15.31) . we obtain 
(5.4) 

^ m 

nj E ^(«)#sEn™(CrJ(A:) = 0FVri/fc,Q£) ® ^”^-VT./fc,Q^). 

dr T[ \\ ) j^p]) d=0 

R is now a matter of combinatorics to prove (15.ID . First we have to rewrite am^{T) and 
am 2 {T). In the odd case, that is for ami(T'), Lemma im ihi and Proposition IB.61 together 
gives 


am, {T) = E • Ci{- 9 i+mi - 2i) • Tr(Frob : 

0<2£<mi 

where gi is the genus of Ct^; rii = 2gi + 1. For the even case, the number am{T) is like 
am{T) — {q + l)am-i(T) if using the definition of the odd case. This gives 

an. 2 {T) = E + "^2 - 2i) ■ TT(Frob : 

0<2i<m2 

-{q + l) E q'-Ci{-gi+m 2 - 2 £-l)-TT{Fioh:H”^^-^^-\jT 2 /-k,Qe))- 
0<2e+l<m2 


Having the expressions of ami (^) (^) hand, one sees that (15.ID follows from 

(15.4D and Corollary IB.51 This finishes the proof of Proposition 15.81 □ 


With the endoscopic transfer formula we had right before Proposition 15.81 it now suf¬ 
fices to show that when we run over all possible choices of coprime separable polynomials 
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PTi{x),pt 2 {x) G k[x] of degree ni and n 2 , respectively, we have 

E E E 7(^mi)Omi(El)am2(72) ' ') ® •) 

spans the linear space of stable distributions supported on the union of all Nm-^ x iVmj 
with mi + m 2 = m. 

In other words we have to prove the vectors {ami (^i)om 2 (? 2 ))mi+m 2 =m for different Ti, T 2 
span \ mi+m 2 =m, o< 2 mi<ni, 0 < 2 m 2 <n 2 }^ That this is always the case can be proved 

with a parity trick similar to Lemma 15.41 This finishes the proof of Theorem 15.71 □ 


Remark 5.11. In fact, it was endoscopic transfer which led us into conjecturing the results 
in Theorem 14.11 and Theorem 14.111 before knowing how to compute them. The point is 
that without having a good method to compute Shalika germs. Section [3] already tells us 
that the stable Shalika germs T(T) should be expressed in terms of linear combinations 

of T^Sym™ {CT){k), 0 < m' < m. Together with Assem’s conjectures, this suggests that 
something like (15.11) . with some a priori unknown coefficients, should be true. 

On the other hand, Nm only exists as a nilpotent orbit with codimension 2m in the 
regular nilpotent orbit if 2m < n. In other words, this suggests that the stable Shalika 
germ formula (which we proved to be am{T)), should be something that vanishes when 
2m > n. This together with some weaker computation was what led us to the formula 
rf (T) = am{T). 


6. Local character expansions of supercuspidal representations 

This section is devoted to the application of our Shalika germ formulas to local character 
expansion of specific supercuspidal representations. Briefly speaking, we use our result 
on supercuspidal representations whose local character looks like the Fourier transform 
of J{T, •) to obtain a Harish-Chandra-Howe local character expansion, and invoke the 
interpretation of character expansion by Moeglin-Waldspurger m- For the use of the 
result in m, we assume char(T) = 0 in this section. 

Again fix T G 0 (l)^®(/c). Recall 0 (l)(fc) = 0(F’)x-i/ 2 / 0 (^)x,o = 0 (^)x,i/ 2 / 0 (-^)x,i- 
Furthermore we have that 0 (l)(/c) is self-dual, allowing us to identify 0 (i^)x,-i/ 2 / 0 (-^)x,o and 
0 (-^)x,i/ 2 / 0 (-^)x,i as the dual of each other. With a choice of non-trivial additive character 
il^ : {k,+) —C^, the elment T then give rises to a character on 0 (i^)x,i/ 2 / 0 (-^)x,i — 
G{F)^ il 2 /G{F)if_^i, and thus a 1-dimensional representation of G(F)x,i/ 2 - We denote by 
this representation. 

The compact induction 

VTT := c-indgJj^^^^V’T = {/ G G^iG{F)) \ /(5152) = V’T(ffi)/(ff 2 ), V51 G G(F),,i/ 2 }. 

can be shown to be the direct sum of finitely many supercuspidal representations. Let 
r be the number of irreducible factor of the characteristic polynomial pt{x) of T and 
L = k[x]/pT{x) be an etale algebra over k; L is the direct product of r finite extensions of 
k. We have Stabo(y)(r) = Res^/X 2 has 2'’ points defined over k. Then ttt is the direct sum 
of 2^ distinct irreducible supercuspidal representations [l5l Prop. 2.4] of depth d. These 
are examples of epipelagic representations of Reeder and Yu [T5] . 
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Now let be the character of Tr^. In other words, G C“(g(F)) is the {G{F)- 
conjugation) invariant distribution such that for any / G C^{G{F)), <h^y(/) ;= Tr(7r7’(/)). 
Here to define TTTif) we need a choice of measure on G{F), which we give in Appendix lAl 

The basic philosophy that goes back to at least Harish-Chandra is that characters should 
be compared with Fourier transforms of orbital integrals. Use as in Appendix the self¬ 
dual structure ')) measure on q{F). This gives, for / G G^{q{F)), its Fourier 

transform 

/(A) := / ^P{B{X,Y))dY. 

We define J{X,f) := J(X,f), the Fourier transform of orbital integrals. Fix a lift 
T G 0 (T)x,i/ 2 - What one has is that 

Lemma 6.1. (i) is supported on G{F)^ i/ 2 - 

(a) Let e : 0(T)x_i/2 ^ G'(-^)x,i/2 a mock exponential map (see [H Hyp. 3.2.1], for 
us it can be given by the Cayley transform). Then for any f G G^{q{F)j^ 1 / 2 ), 

0^^(/oe) = 2'-. J(r,/). 

(Hi) For each of the 2'’ components of-KT, its character (which has larger support), when 
restricted to G'(T)xp /2 and pulled back to q{F)^ i /2 via e, is equal to J{T,f). 


From now on let nf be any fixed component of ttt, and its character. Let 0(0) be 
the set of nilpotent orbits. Then the Harish-Chandra-Howe local character expansion [HI 
Thm. 4] states that there exists constants {co{'^t))o&O{ 0 ) ^ such that 

(6.1) ck^.(/oe)= co{7rf)J{OJ), 

c>eO(o) 

for all / that are supported in a sufficiently small neighborhood U C 0(T)xp/2 of 0 G 
^(T). On the other hand, in [12j Moeglin and Waldspurger proved that, if O' is any nilpotent 
orbit satisfying that for any O whose boundary contains O' we have co = 0, then cc)'(vr^) 
is equal to the dimension of the degenerated Whittaker model associated to O' of Trf. 

Now, restricting to the small neighborhood U, we have by Lemma lb.H iii) and (|6.1|1 that 

J(r,/)= ^ co{7rf)JiOJ). 

oeO{o) 

By inversing the Fourier transform, we see 

Corollary 6 . 2 . We have co{t^j) = To{T), the latter are given by formulas in Theorem 
UJ^ and UA^ 

Corollary 6 . 3 . For any m > 0 we can find G = G(m,q) such that for any n > G, we 
can find supercuspidal representations of Un{E/F) of the form irf such that co{t^t) = 0 
for every nilpotent orbit O of two Jordan blocks of sizes n — m' and m', 0 < m' < m. Here 
by abuse of language the m' = 0 case corresponds to an orbit with a single Jordan block, 
namely a regular nilpotent orbits. 

Proof. Take T so that pt{x) has as many irreducible factors as possible, so that # Jt[2] (k) = 
2^ with r > i_|_iog The varieties in Theorem 14. 101 and 14.131 are JT[2]-covers of Sym™ (Ct) 
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(and Sym™ (C^), etc), whose numbers of points can be bounded by the Weil bound on the 
Frobenius trace. Now for any J'r[2]-cover of Sym"* {Ct), the hber above a rational point 
in Sym”^ (C'r) is a J'r[2]-torsor. Recall that the orbits in the stable orbit of T are classified 
by H^{k, JtI^]), and when T runs over all such orbits in the same stable orbit, the fiber 
above any chosen rational point in Sym™' {Ct) will also run over all possible JT[2]-torsors. 

Once n is large enough, we have r large enough so that 2’’ will be much greater than 
X)m'=o (C'r)(fc) (and more for other covers and covers of Sym™' (C^), etc). We 

can thus find an orbit in the stable orbit of T, i.e. a class in H^{k, such that 

for the corresponding covers Sym {Ct) in Theorem I4.1UI and 14.131 0 < m' < m, the 
torsor above each rational point is non-trivial. That is to say Sym (Ct) (and similarly 

-- m' -- fn',* 

Sym (C'^),Sym {Ct), etc) has no rational points. When n is odd this says co{t^t) ~ 

^o{T) = 0, which is what we want. When n is even we have instead co(7ry)+CAd(?i)o('^T) = 
To{f) + rAd(«)ci(T’) = 0. Since inductively by [l2] we have cc)(7rf.), CAd(«)c>(7rr) > 0, we 
conclude that they all vanish. □ 


Note a nilpotent with two Jordan blocks is never in the closure of a nilpotent orbit with 
more than two Jordan blocks. One can thus have many examples where the dimension of 
the degenerate Whittaker models are (up to constant) number of rational points on varieties 
in Theorem 14.101 and 14.131 

Example 6.4. For example, take n = 2g + I odd and take pt{x) G k[x\ any polynomial 
of degree 2g -\- 1 that is the product of r distinct irreducible factors with r > 1. Let 
Ct = {y^ = Pt{x)) (the smooth completion). Take an etale Galois Jt [2]-cover Ct of Ct 
for which the fiber above oo G Ct is a non-trivial JT[2]-torsor. Such a choice corresponds 
to an orbit of such T in its stable orbit. The corresponding representation has Cfj^{'K^) = 

^j^:^py#Sym°(C'r)(A:) = 0 and c^j(vrf.) = ^j^y 2 ](k) #^T{k), i.e. the dimension of the 

degenerate Whittaker model for the subregular orbit Ni is times the number of 

rational points on Ct, a curve of genus 2^^{g — 1) -|- 1 over k. It will be interesting to see 
how these points actually “live” on the degenerate Whittaker model. 


Appendix A. Normalization of measures 

This appendix is for the normalization of semisimple and nilpotent orbital integrals on 
our p-adic group G. Our normalization essentially follows that of |12j . 

For X G q{F) regular semisimple, our J{X, ■) is what is usually written |iJ(X)|^/^^j^(-). 
More precisely, let D{X) := det(ad(X)|g/g^), where denotes the centralizer of X. The 
norm | • | on T is such that |7r| = q~^- We define 

J(X,/):=|Z1(A)|V2 f f{Ad{g)X). 

Jg{f)/g^{f) 

And the normalization of measures goes as follows. Fix an additive character ip ■. F ^ 
such that Ip is trivial on Fp but not on Op. Let B{-, •) ^ g x g —>■ Gq be an T-Killing form 
on g. In fact in the article we’ll take B{-,-) to be the naive trace form on the space of 
anti-hermitian spaces, which has the property that for any point x' on the building and 
d G M, ip{B{-, ■)) identifies g(F’)x',d:d+ as the dual of g(T’)x/_rf:(_rf)+. 
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The Haar measure on q{F) is taken to be the one that is self-dual by V’O-B, and the Haar 
measure on G{F) to be the one so that the (mock) exponential map is measure preserving 
near the identity, C g is a subspace on which is non-degenerate, and the Haar 

measure on Qx{F) and G^^F) is defined in the same way by restricting to gj^ x Sx- 

This defines the required Haar measure in the above regular semisimple orbital integral. 

Lastly, the normalization of nilpotent orbital integrals goes as follows. We assume in this 
article that char(T) = 0 or char(T) > n. This implies that any nilpotent orbit O C g is 
smooth with expected tangent space; for N G O, we have T^O = q/qn- Now g/gAr has a 
symplectic structure B^ : {X,Y) i-)- B{[X,Y], N). 

We take the measure on O to be given by the top wedge power of this symplectic 
form. More precisely, this measure has the following interpretation. Take a Lagrangian 
F-subspace L C g(F)/gAr(F) and C L any lattice. Let L' be any F-complement of L 
and = {X G L' | 'iP{Bn{X,Y)) = 1, VT G A^} be the dual lattice. Then A^ -|- is 
assigned to have measure 1. 


Appendix B. Catalan numbers 

This appendix discusses combinatorics that appear in analyzing Shalika germs and their 
endoscopic transfer consequence. We omit the proofs, which are fairly elementary. 


Definition B.l. For any integer i > 0, we define degree I polynomials Cfix) G Q[x] by 


GAx) = 


X 


ix + 2£)-ill\ 


J^(x + £ + i). 


Remark B.2. GfiO) = 0 except for C'o(x) = 1. Also Cfil) is the classical Catalan numbers 
1,1,2,5,14,.... See e.g. Wikipedia. 


Proposition B.3. For any integer £> Q, Cfix -|- 1) — Cfix) = Q_i(x -|- 2). 


The following observation was shown to me by Joel B. Lewis. 
Proposition B.4. Let G{x,q) := Y^^QCAx)q^, we have 

T-Vr^' 


C{x,q) = 


2q 


Corollary B.5. We have G{x+y,q) = G{x,q)G{y,q). Equivalently Cfix+y) = GAx)CQ{y)+ 
Q_i(x)Ci(y) + ... + Co(x)Q(y). 


Proposition B.6. Let A = Ihe lower triangular matrix with entries in 

Q[x,g] with 


Aij — 




if i — j Y 2i, I G Z>o- 


0 


otherwise. 
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Then the inverse of A is given by 

(q^Ci{-x + j) ifi = j + 2 £, £ £ Z>o. 

= 

I 0 otherwise. 


We add another vaguely related proposition, which is used in the end of subsection 14.21 


Proposition B.7. Let 0 < m' < m < g. Let '^rn,m' T Sm be the subset of bijections of 
,m} that satisfies an equivalent of Condition \4.9\ a{i) = i for i = and if 

either j <m and j — i = 2 , or j > m and j — i = 1 , then we have a{j) > (T{i). 


Write 63 ( 0 -) = #{1 < i < g \ a{i) > a{i + 1 )}. Then 63 (a) < J a € and 

forO<r< 

''g — m' 
r 


£ ^rn,m' \ 63(0') ^ — 


Appendix C. Restrictions on characteristic of local and residue fields 

In this appendix we explain what restrictions are necessary, and why some others can 
be relaxed. Recall F is the local field and k its residue field. The restriction we have for 
the results in this paper is char(A;) 7 ^ 2 and either char(R) = 0 or char(R) > n, where 
G = Un(E/F) (except for subsection 15.21 and Section [ 6 l in which we furthermore require 
char(R) = 0). Our main reference here is |19l Appendix A]. To begin with, if char(T’) 7 ^ 0, 
then for well-definedness of orbital integrals, finiteness of nilpotent orbits and the validity 
of the theorem of Shalika dni), we need char(F) > n and [171 III. 4.14]. However as our 
G/ E — GL„, we can check that [TTl III. 4.14] is valid as long as char(F) > n. 

Now we discuss the assumption on char(A;). The restriction char(A:) 7 ^ 2 is used ev¬ 
erywhere; we don’t bother to deal with quadrics over F 2 and wildly ramified group, etc. 
The only place that we need to assume more is Hypothesis 14.21 where we use DeBacker’s 
homogeneity result, whose assumption on char(A:) we don’t know how to avoid. However 
one can do the following: once we establish the result in Section |4] in the case char(/c) 3> 0, 
we can compare the result with the method in m- Roughly speaking, the method in m 
computes Shalika germs in terms of the same varieties in Section 1,4.21 but with (in general) 
uncontrollable combinatorics. 


Let’s take Theorem 14.101 as an example. The method in m will compute T(T) also 

in terms of #Sym (CT)(k), m' < m, but with unknown coefficients Pm,m'(Q,g) £ Q(^)[ 5 '] 
that are polynomial in the genus g and rational in q, independent of the choice of F, k 
and n = 2g + 1. Given that we already know Theorem 14.101 for char (A:) 3 > 0, we know the 
method in m must give us the same result. 


This reduces the restriction on char (A:) to only the restrictions that we need in [19], which 
assumes char(A:) 7 ^ 2 because we have a Z/ 2 -grading on G, and assumes (char(A:),n) = 1 
for [191 Claim 2.4]. However, what is actually needed for the latter is an self-dual structure 
on g = g[„/fc, which we do have regardless of char(A:). In fact, even if G = SUn(E/F) and 
0 = sin, we are still good, as one can work with g* = pg[„ for the need of [l9l Claim 2.4]. 
In any case, we can drop the assumption (char(A;),n) = 1. 


There is also m Hypothesis 3.1] which is only known to be true for general groups 
assuming char (A:) large. However in our case |19[ Hypothesis 3.1] is exactly verified by the 
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bijection between nilpotent orbits in q{F) and nilpotent orbits in g(l)(A:) described in the 

beginning of Section [H In conclusion, we can work with any char(A:) odd. 
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